Live data from Hacker News

Versioning data in Postgres? Testing a Git like approach

specfy.io

21–30 of 92 posts

Re: Versioning data in Postgres? Testing a Git like approach

#21
post #16

I didn’t see it in the article but that’s exactly what https://postgres.ai/ is doing already unless I’m missing some thing.

I haven’t read about NeonDB before but a brief search of their docs doesn’t mention ZFS. Postgres.ai uses ZFS to implement the snapshots, branching, etc.

I’d be very interested in a comparison of the two.

Re: Versioning data in Postgres? Testing a Git like approach

#24
4 years ago I wrote an internal tool to serialize records of our hierarchical database to a bunch of hierarchical json files, which are then managed in a normal git repo.

This was/is a very specific solution to our very specific set of problems. So not applicable to the general problem of "versioning a database".

It is still in use and still under active development – I'm actually fixing a few bugs with it just now.

I wrote much more about it here:

https://news.ycombinator.com/item?id=25005993

Re: Versioning data in Postgres? Testing a Git like approach

#25
post #14

Better idea: just install the Postgres extension that enables ISO SQL Temporal Tables, just like what MSSQL and MariaDB have had for 8 years now: https://pgxn.org/dist/temporal_tables/

author here: Yes I agree, and I mention it at the end. However it's not possible to install the extension everywhere, for example it's not available in GCP Cloud SQL unfortunately

I feel your pain, except in my case it's the crippling inability to use SQLCLR in Azure SQL, so we can't even do things like use a Regex to clean-up data in-situ. Yeargh.

Re: Versioning data in Postgres? Testing a Git like approach

#26
post #2

I just taught one of my developers how to use a basic Oracle Flashback query to pull a historical version of a table. This is my cheat sheet: ALTER SESSION SET NLS_TIMESTAMP_FORMAT = 'YYYY/MM/DD HH24:MI:SS'; select count(*) from dual AS OF TIMESTAMP TO_TIMESTAMP('2010/01/01 00:00:00'); I was aware that Postgres had "time travel," but I don't know if it used this syntax, but I am aware that the feature has been remove…

Oracle, with all the baggage attached to the namesake company, is a remarkable database. I was working in a dual MySQL/Oracle environment 20 years ago exactly, and MySQL - still at version 3 o 4 - was a toy DB, compared. I was writing Oracle queries that could make your head spin, with optimizer hints, for example: https://renenyffenegger.ch/notes/development/databases/Oracl...

> I was writing Oracle queries that could make your head spin

Absolutely, and some of the things Tom Kyte (Oracle's resident DB performance guru) could do with Oracle were spectacular.

As you say, its a shame about the Oracle baggage, and in particular the steep price tag. Otherwise I'm certain it would be far more widely deployed as database.

Re: Versioning data in Postgres? Testing a Git like approach

#27
post #9
post #2

I just taught one of my developers how to use a basic Oracle Flashback query to pull a historical version of a table. This is my cheat sheet: ALTER SESSION SET NLS_TIMESTAMP_FORMAT = 'YYYY/MM/DD HH24:MI:SS'; select count(*) from dual AS OF TIMESTAMP TO_TIMESTAMP('2010/01/01 00:00:00'); I was aware that Postgres had "time travel," but I don't know if it used this syntax, but I am aware that the feature has been remove…

Expanding on the the above, > The original version of PostgreSQL from the 1980s did not remove dead tuples. The idea was that keeping all the older versions allowed applications to execute “time-travel” queries to examine the database at a particular point in time [via https://ottertune.com/blog/the-part-of-postgresql-we-hate-th... ] Postgres deprecated support for time-travel in ~1997 and the more general notion of…

> (bi-)temporal versioning

I once spent some time trying to find a way to do bi-temporal versioning in Postgres.

The only thing I found was a half-dead abandonware external project and an associated presentation PDF from some conference the author once spoke at.

I was unaware that they previously had some form of temporal queries and deprecated it. That is a great shame.

Re: Versioning data in Postgres? Testing a Git like approach

#28
post #2

I just taught one of my developers how to use a basic Oracle Flashback query to pull a historical version of a table. This is my cheat sheet: ALTER SESSION SET NLS_TIMESTAMP_FORMAT = 'YYYY/MM/DD HH24:MI:SS'; select count(*) from dual AS OF TIMESTAMP TO_TIMESTAMP('2010/01/01 00:00:00'); I was aware that Postgres had "time travel," but I don't know if it used this syntax, but I am aware that the feature has been remove…

Oracle, with all the baggage attached to the namesake company, is a remarkable database. I was working in a dual MySQL/Oracle environment 20 years ago exactly, and MySQL - still at version 3 o 4 - was a toy DB, compared. I was writing Oracle queries that could make your head spin, with optimizer hints, for example: https://renenyffenegger.ch/notes/development/databases/Oracl...

Using hints is typically a sign of failure. The DB optimiser should typically not need them.

Re: Versioning data in Postgres? Testing a Git like approach

#30
I would prefer a generalized version of the "Oracle Flashback" feature: ability to add extra where conditions to a table, e.g.

create table test (id int primary key, attr int, deleted_at timestamp);

alter table test add default_filter is_active where deleted_at IS NULL;

then all selects like "select * from test where attr = 42" automatically gets added the above criteria and becomes "select * from test where attr = 42 AND deleted_at is NULL"

When you want old rows, you can override the default condition using something like:

select * from test where attr = 42 filter is_active (deleted_at IS NULL or deleted_at > '2010-01-01'::datetime)

You would then be able to have multiple criterias instead of just when row was deleted.

Post reply on HN