Live data from Hacker News

Temporal Tables PostgreSQL Extension

github.com

21–28 of 28 posts

Re: Temporal Tables PostgreSQL Extension

#21
post #6

Earlier quoted context omitted.

(Disclaimer: I'm coming from MariaDBs temporal table feature but this is basically the same in PostgreSQL) Temporal tables adding an additional "time axis" to SQL databases. A "valid from" and "valid to" field is added to each row and the SQL syntax is extended for allowing two new types of queries: 1. Query the data as of a specific point in time. E.g. "show all customers as of April 9th 2021". This not only limits…

> Those validity dates represent a period in the real world. We have tons of that at work, so this feature would have been nice. Currency exchange rates, dozens of official code lists (including countries!), VAT registration status of companies. If a user makes a change to a declaration submitted at an earlier date, then the data from the original submission date must be used, so we need to keep all this around. Alas…

MariaDB and SQL Server both have equivalent features, for what it’s worth.

Re: Temporal Tables PostgreSQL Extension

#22
post #21

Earlier quoted context omitted.

> Those validity dates represent a period in the real world. We have tons of that at work, so this feature would have been nice. Currency exchange rates, dozens of official code lists (including countries!), VAT registration status of companies. If a user makes a change to a declaration submitted at an earlier date, then the data from the original submission date must be used, so we need to keep all this around. Alas…

MariaDB and SQL Server both have equivalent features, for what it’s worth.

Thanks, nice to know!

I know we have plans to start adding SQL Server support this year (due to customer demand), and we might end up doing a full transition. Stuff like this certainly doesn't make that case weaker.

Re: Temporal Tables PostgreSQL Extension

#23
post #5

What problem does this solve?

In short, it lets you time-travel to retrieve prior states in tabular data.

One useful example is continuous forecasting. Every forecast is done with a certain moving window of historical training data, and each time the forecast is run, this training data is different.

In order to evaluate/backtest the performance of the forecasting algorithm over time, you need to be able to retrieve all the previous training datasets from the database, calculate a performance metric, and then aggregate. This lets you do that on a continuous basis in the database itself, without dumping each training set to secondary storage each time.

Also, in many real life applications, historical data isn’t immutable — data corrections can arrive after the fact that will change state, so it’s usually not sufficient to just use a simple WHERE clause to retrieve a particular time range. This is where time travel becomes really useful.

Re: Temporal Tables PostgreSQL Extension

#24
I've used this a couple times both with the regular implementation and the version that can run in AWS managed databases.

It is surprisingly useful. Many use cases benefit from having a full history that can be instantiated at any time. In particular, you can apply migrations to these tables which means your old data is still in the correct format when you need it.

Re: Temporal Tables PostgreSQL Extension

#25

I was part of a team at NearForm using this for a project on an EC2 instance. In order to move to AWS RDS we had to recreate the functionality of temporal_tables as a PostgreSQL function, rather than extension. When we switched, we found that although there were minor bugs, we didn't have any noticeable loss of performance and we have used it ever since for many projects. https://github.com/nearform/temporal_tables I…

I see you have a “nochecks” version, what would be its use case?

Re: Temporal Tables PostgreSQL Extension

#26
post #25

I was part of a team at NearForm using this for a project on an EC2 instance. In order to move to AWS RDS we had to recreate the functionality of temporal_tables as a PostgreSQL function, rather than extension. When we switched, we found that although there were minor bugs, we didn't have any noticeable loss of performance and we have used it ever since for many projects. https://github.com/nearform/temporal_tables I…

I see you have a “nochecks” version, what would be its use case?

It says right there in the readme: performance.

> This version is 2x faster than the normal one, but more dangerous and prone to errors.

Re: Temporal Tables PostgreSQL Extension

#27
I'm currently getting started with TimescaleDB for storing lots of time-series data (all the public data from cryptocurrency exchanges).

Turns out, there's a bit of weird stuff required in order to collect orderbook data, which involves recording updates, but also the 'full state' (or at least as much as I can get) periodically. The current plan is to add an interface between the DB and the consumers that makes it work a bit like a temporal table (if we're storing the full state at the beginning of every hour, the previous state along with any updates should expire at the end of the hour).

If postgres could also leverage the temporal tables extension we might prevent needing to store a lot of entries that 'invalidate' previous entries, but then I imagine there's internal overhead with the temporal tables extension as it manages expiry time.

I'm wondering, are there any notes on using the temporal tables extension with the timescaledb extension? Wondering if that's something worth investigating.

Post reply on HN