151–157 of 157 posts
Earlier quoted context omitted.
You don’t do deletes. The log is append only. If you want to get rid of an entry, you create a new entry that negates the existing entry.
Yes, but if order doesn't matter, the delete might appear before the create. Might be misunderstand something
Earlier quoted context omitted.
I believe "ledger" implies commutative property (order does not matter).
I am not aware of any such implicit connection of ledger and commutative property, also couldn't find anything as my google-fu is letting me down. Anything I can refer to? Generally curious to know use of term ledger outside of accounting and blockchains. I have seen it used to mean WAL before, so I am taking this with a dose of skepticism.
Depending on the data model of your log, if calculating the current state is a commutative operation, I think it's fair to call it a "ledger".
Earlier quoted context omitted.
Yes, exactly right. One log per logical entity, here "payment ID". The way our open source project implements that is with a partitioned log and indexes at key-granularity, so it is like virtually a log per key.
How do you deal with one side of transaction getting lost? Traditional double entry accounting puts entry for each side of the transaction. It will cause problems if one side is lost or delayed. How do you handle transactions in general? It is pretty common to have multiple changes that need to be applied together. In a single log, can write the changes and then write a record that says the changes are committed. Thi…
This is a basic concept in accounting. The general ledger is an immutable log of transactions. Other accounting documents are constructed from the general ledger, and can, if necessary, be rebuilt from it. This is the accepted way to do money-related things. Synchronization is called "reconcilation" in accounting terminology. The computer concept is that we have a current state, and changes to it come in. The databas…
- General legers are formed by way of transactions recorded as journal entries. Journal entries are where two or more accounts from the general ledger are debited & credited such that total debits equals total credits. For example, a sale will involve a journal entry which debits cash or accounts receivable, and credits revenue.
- The concept of the debits always needing to equal credits is the most important and fundamental control in accounting. It's is the core idea around which all of double entry bookkeeping is built.
- temporally ordered Journal entries are what form a log from which a general ledger can be derived. That log of journal entries is append-only and immutable. If you make an mistake with a journal entry, you typically don't delete it, you just make another adjusting (i.e. correcting) entry.
Having a traditional background in accounting as a CPA, as a programmer I have written systems that are built around a log of temporally ordered transactions that can be used to construct state across time. To my colleagues that didn't have that background they found it interesting but very strange as an idea (led to a lot of really interesting discussions!). It was totally strange to me that they found it odd because it was the most comfortable & natural way for me to think about many problems.
Excuse me for sounding rough, but - isn't this reinventing comp-sci, one step at a time? I learned about distributed incrementally -monotonic logs back at the late 90s, with many other ways to do guaranteed transactional database actions. And I'm quite certain these must have been invented in the 50s or 60s, as these are the problems that early business computer users had: banking software. These are the techniques t…
I think your points are pretty spot on - most things have already been invented, and there's too much of a move-fast-and-break-things mentality. Here's a follow-up thought: to what extent did the grey-beards let us juniors down by steering us down a different path? A few instances: DB creators knew about replicated logs, but we got given DBs, not replicated log products. The Java creators knew about immutability: "I…
I wish James had been in favor of immutability in designing java.util.Date!
This post makes a great case for how universal logs are in data systems. It was strange to me that there was no log-as-service with the qualities that make it suitable for building higher-level systems like durable execution: conditional appends (as called out by the post!), support very large numbers of logs, allow pushing high throughputs with strict ordering, and just generally provide a simple serverless experien…
> It was strange to me that there was no log-as-service...
Any blockchain that is built upon PBFT derivative is such a system.