Live data from Hacker News

MariaDB Temporal Data Tables

mariadb.com

41–44 of 44 posts

Re: MariaDB Temporal Data Tables

#41

Earlier quoted context omitted.

Change tracking is not a fully bitemporal scheme, though. A bitemporal table tracks _two_ timelines. One is about when facts in the world were true ("valid time" or "application time"), the other is about the history of particular records in the database ("transaction time" or "system time"). Change tracking can only capture the second.

This was extremely confusing to me on first read but, since temporal data is an area of interest to me for improving an existing service I maintain as well as for future endeavors, it prompted me to go do a bunch of reading to understand what was meant by the distinction. If others are reading this and confused, maybe what I learned will help make the distinction clearer. There’s two cases where valid/applicable time…

It is hard to maintain the two timelines in one's mind at once. Most of the time I maintain the dehydrated version that "you need both" and talk myself through the details when I need them.

Typically what happens is that folks incorrectly capture parts of both timelines. For "valid time", folks will have "created at", "deleted at", "updated at" fields etc and assume this covers it. Unfortunately, it doesn't really capture proper spans. If I have a fact that starts being true on Tuesday, then add another one that starts being true on Thursday, can I deduce that the first fact was true from Tuesday until Thursday?

No, I can't logically make that claim. It's possible that the first fact was true on Tuesday only, leaving a gap on Wednesday. Without explicit valid time, I can't find those kinds of holes, or apply constraints against them.

Similarly, folks rely on history tables, audit tables etc to try to capture transaction time changes. These are still not enough, for the same reason. You need the span of time, the interval, to properly assert the time during which the database held a record to be true. When we discover that the Thursday fact was wrong, we need to be able to know over what window we held that to be true. Overwriting the record, or entering an update into a log etc, is not enough to truly reconstruct the history. You must be explicit about the span.

The necessity of bitemporalism was easy for me, because I had been responsible for either creating or maintaining multiple databases in which I could not answer a wide range of questions after the march of time. I learned many of the standard hacks (copy the price of a stocked item into the invoice line item, or it will change beneath you! Use slowly changing dimensions! Have a transactions table which creates a slow-as-hell copy of what the database does internally, but not as well!). When I read Snodgrass's book it all went clunk and I've been a wild-haired proselyte ever since.

Re: MariaDB Temporal Data Tables

#42
post #20

How does this feature compare to MS SQL's Temporal Tables https://docs.microsoft.com/en-us/sql/relational-databases/ta... ? This feature seems to be well fitted to support some of the cases where event sourcing is introduced, I wonder if someone successfully applied event sourcing with use of temporal tables to reduce the amount of work that has to be done in the application code (Akka, etc.).

When we looked at temporal tables in SQL Server for event sourcing, I was put off by the fact that you have to read from multiple tables. CDC + some external data source still seems to be the better solution here, imo.

What do you use for your event sourcing? Do you use Akka/Akka.NET Persistence or some other application framework?

Re: MariaDB Temporal Data Tables

#43
post #20

Earlier quoted context omitted.

When we looked at temporal tables in SQL Server for event sourcing, I was put off by the fact that you have to read from multiple tables. CDC + some external data source still seems to be the better solution here, imo.

What do you use for your event sourcing? Do you use Akka/Akka.NET Persistence or some other application framework?

We use Debezium and Kafka, it's been really nice.

Re: MariaDB Temporal Data Tables

#44
post #43

Earlier quoted context omitted.

What do you use for your event sourcing? Do you use Akka/Akka.NET Persistence or some other application framework?

We use Debezium and Kafka, it's been really nice.

Do you just process events it for analytics or do you actually perform rehydrating application state or replicating application state?
Post reply on HN