Earlier quoted context omitted.
(Disclaimer: I'm coming from MariaDBs temporal table feature but this is basically the same in PostgreSQL) Temporal tables adding an additional "time axis" to SQL databases. A "valid from" and "valid to" field is added to each row and the SQL syntax is extended for allowing two new types of queries: 1. Query the data as of a specific point in time. E.g. "show all customers as of April 9th 2021". This not only limits…
> Those validity dates represent a period in the real world. We have tons of that at work, so this feature would have been nice. Currency exchange rates, dozens of official code lists (including countries!), VAT registration status of companies. If a user makes a change to a declaration submitted at an earlier date, then the data from the original submission date must be used, so we need to keep all this around. Alas…
Temporal Tables PostgreSQL Extension
21–28 of 28 posts
Re: Temporal Tables PostgreSQL Extension
#22Earlier quoted context omitted.
> Those validity dates represent a period in the real world. We have tons of that at work, so this feature would have been nice. Currency exchange rates, dozens of official code lists (including countries!), VAT registration status of companies. If a user makes a change to a declaration submitted at an earlier date, then the data from the original submission date must be used, so we need to keep all this around. Alas…
MariaDB and SQL Server both have equivalent features, for what it’s worth.
I know we have plans to start adding SQL Server support this year (due to customer demand), and we might end up doing a full transition. Stuff like this certainly doesn't make that case weaker.
Re: Temporal Tables PostgreSQL Extension
#23What problem does this solve?
One useful example is continuous forecasting. Every forecast is done with a certain moving window of historical training data, and each time the forecast is run, this training data is different.
In order to evaluate/backtest the performance of the forecasting algorithm over time, you need to be able to retrieve all the previous training datasets from the database, calculate a performance metric, and then aggregate. This lets you do that on a continuous basis in the database itself, without dumping each training set to secondary storage each time.
Also, in many real life applications, historical data isn’t immutable — data corrections can arrive after the fact that will change state, so it’s usually not sufficient to just use a simple WHERE clause to retrieve a particular time range. This is where time travel becomes really useful.
Re: Temporal Tables PostgreSQL Extension
#24It is surprisingly useful. Many use cases benefit from having a full history that can be instantiated at any time. In particular, you can apply migrations to these tables which means your old data is still in the correct format when you need it.
Re: Temporal Tables PostgreSQL Extension
#25I was part of a team at NearForm using this for a project on an EC2 instance. In order to move to AWS RDS we had to recreate the functionality of temporal_tables as a PostgreSQL function, rather than extension. When we switched, we found that although there were minor bugs, we didn't have any noticeable loss of performance and we have used it ever since for many projects. https://github.com/nearform/temporal_tables I…
Re: Temporal Tables PostgreSQL Extension
#26I was part of a team at NearForm using this for a project on an EC2 instance. In order to move to AWS RDS we had to recreate the functionality of temporal_tables as a PostgreSQL function, rather than extension. When we switched, we found that although there were minor bugs, we didn't have any noticeable loss of performance and we have used it ever since for many projects. https://github.com/nearform/temporal_tables I…
I see you have a “nochecks” version, what would be its use case?
> This version is 2x faster than the normal one, but more dangerous and prone to errors.
Re: Temporal Tables PostgreSQL Extension
#27Turns out, there's a bit of weird stuff required in order to collect orderbook data, which involves recording updates, but also the 'full state' (or at least as much as I can get) periodically. The current plan is to add an interface between the DB and the consumers that makes it work a bit like a temporal table (if we're storing the full state at the beginning of every hour, the previous state along with any updates should expire at the end of the hour).
If postgres could also leverage the temporal tables extension we might prevent needing to store a lot of entries that 'invalidate' previous entries, but then I imagine there's internal overhead with the temporal tables extension as it manages expiry time.
I'm wondering, are there any notes on using the temporal tables extension with the timescaledb extension? Wondering if that's something worth investigating.
Re: Temporal Tables PostgreSQL Extension
#28Codesleuth’s comment: https://news.ycombinator.com/item?id=26768220