Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Books: an immutable double-entry accounting database service

developer.squareup.com

Post reply on HN
Print view View original

61–66 of 66 posts

Re: Books: an immutable double-entry accounting database service

#61

Post by WorldMaker »

inopinatus wrote: post #37
dragonwriter wrote: post #17

This looks good but some of the background has the air of the kind of artificial problems presented in late night infomercials: > It’s impossible to split a single payment into multiple payouts, since there is a many-to-one relationship of payments to payouts. So your model used a many-to-one relationship when you really wanted a many-to-many? > Since this is just a SQL database, there’s nothing preventing the payout…

I think this mocking of their intention and problem statements is misplaced. > So your model used a many-to-one relationship when you really wanted a many-to-many? In many schema discussions you’d be right, but in the case of payments systems it’s not the case. Payments and payouts are separate transactions and only loosely related. The allocation of credit from one leads to an increment in the settlement of the othe…

>> You mean, like a NOT NULL constraint? Or, if you mean zeroed out, an appropriate constraint and/or before update trigger protecting the amount?

> When you want to archive off aged records, or constrain across db shards, this is a serious issue. Taking the piss doesn’t make it go away. But in that particular paragraph I believe they’re referring to the payout itself become void e.g. due to a reversal, and having designed around similar issues I understand their concern. It’s exactly where single-column ledgers become a huge pain in the audit.

My impression from the article was more the "baby record" problem: not everything has been paid yet. They were using NULL to indicate "not paid" and thus couldn't put a NOT NULL constraint on that column because it was a valid value for new rows. There are ugly workarounds that could have allowed them to add NOT NULL such as using a special/magic "guard" payment that indicates not paid instead. (Going down that route you'd probably want a "guard" payment per client and then you're getting close to a backwards form of double-entry accounting.)

Re: Books: an immutable double-entry accounting database service

#62

Post by WorldMaker »

andriosr wrote: post #53

Great write up, but I don’t see how it could scale with the pessimistic lock of Spanner. Updating the current balance of big merchants, which are receiving multiple payments per second seems to create a lot of lost transactions, with the locking of the balance. That would generate deadlocks in RDBMSs, curious to know if Spanner is able to scale it. I built a similar system using DynamoDB and optimistic locking, and f…

That is the kind of content I was hoping for in this article. The review of double-entry accounting is nice and all, but the scale details would have been nice beyond hand-waving "we left it to Spanner, and so we followed Spanner's rules".

Similarly a quibble about the "immutable" in the headline given that they are still using a mutable field at the book level for balances to make them easier to query. I guess I was hoping they'd have solved a nicer merkle tree balancer or something for balances. Not that I was hoping for a "blockchain" solution, but something smarter than blockchain for "centralized" trust environments like Square.

Re: Books: an immutable double-entry accounting database service

#63

Post by josephg »

elvinyung wrote: post #48
josephg wrote: post #41

Earlier quoted context omitted.

Huh? Am I missing something obvious here? Plenty of databases support ACID guarantees across multiple machines in a cluster. FoundationDB and CockroachDB come to mind, but I’m sure there are others. What you’ve built is cool, but I’m still confused why existing tools aren’t good enough. From the Cockroach documentation (source: https://www.cockroachlabs.com/docs/stable/transactions.html ) > CockroachDB supports bundl…

Right -- that's why I said most RDBMS. To get ACID and CAP-consistent (which, as an aside, are not the same thing) cross-shard transactions requires some form of migrating your production system onto such a distributed-capable RDBMS that you describe (of which, Spanner arguably fits the description of).

The thing I'm confused about is why migrating your production system onto a custom in-house data store (that you've written and you maintain) is easier / less work than migrating onto something you can use off the shelf which solves the problem you have.

I hear the argument that migrating from (say), postgres -> postgres + books is easier than moving everything into cockroachdb. But why is postgres + books easier than postgres + cockroach? The latter doesn't require you to write your own database from scratch. (And writing your own database is fun, so if you're doing it for that reason I understand, but thats not what the post says.)

Edit: Ah, I see - my confusion was because I missed the part which mentioned that this is just built on top of spanner. That makes sense!

Re: Books: an immutable double-entry accounting database service

#64

Post by elvinyung »

josephg wrote: post #63
elvinyung wrote: post #48

Earlier quoted context omitted.

Right -- that's why I said most RDBMS. To get ACID and CAP-consistent (which, as an aside, are not the same thing) cross-shard transactions requires some form of migrating your production system onto such a distributed-capable RDBMS that you describe (of which, Spanner arguably fits the description of).

The thing I'm confused about is why migrating your production system onto a custom in-house data store (that you've written and you maintain) is easier / less work than migrating onto something you can use off the shelf which solves the problem you have. I hear the argument that migrating from (say), postgres -> postgres + books is easier than moving everything into cockroachdb. But why is postgres + books easier tha…

Yep :P

For a fun fact, CockroachDB was actually started by ex-Squares, so we're definitely very familiar with this exact same argument internally, back in the day when neither CockroachDB nor Spanner existed as a viable option :)

Re: Books: an immutable double-entry accounting database service

#65

Post by codedokode »

nevi-me wrote: post #45
codedokode wrote: post #43

What's good in immutability? If you have found that you have entered too much income and want to lower it, what do you do?

You reverse it with an adjusting entry. What happens if you've transferred too much from your savings account into your current/cheque account? You make another transaction to adjust the initial one.

But then the taxation service can find out that you have lowered your profit to pay less taxes. So what's good in immutability?

Re: Books: an immutable double-entry accounting database service

#66

Post by sudhirj »

painted wrote: post #55
sudhirj wrote: post #27

Earlier quoted context omitted.

Doesn't necessarily need to be hosted, although given that people are fine with running Quickbooks or any other SaaS it's hard to see why they wouldn't be fine with ledger SaaS. Can still make a case for licensing, though, depending on the features. We built this for our internal accounting, it's a ledger service that enforced immutable double entry rules and can run as a container with a Postgres DB configured. http…

do you also have some kind of frontend for it? how do you see what's the flow of money and reporting? does it work with kibana out of the box?

Each reporting system / products does its own reporting, and no plans of adding ELK integration right now, although we are moving to the ELK stack so it might happen.

We think of it more as a database++ with financial primitives baked in, so we want to keep it very simple and dumb. It’s not a complete solution - it’s a book to write in, and you need another service that decides what and how to write and writes it (we call it the bank or teller service, and it’s all business logic for our business). Each product has its own bank to teller service, but they all use the same ledger underneath.

Post reply on HN