Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reconciliation is a knapsack problem

moderntreasury.com

Post reply on HN
Print view View original

11–20 of 57 posts

Re: Reconciliation is a knapsack problem

#11

Post by jvanderbot »

matharmin wrote: post #2

This feels like a marketing piece without any good content. The way it's described, it's not a knapsack problem at all. The knapsack problem is to maximize the total value of the items you fit into the container. In reconciliation, you presumably want to get the best matching between transactions, which is not defined here, and in any case is a completely different problem. Ignoring the knapsack comparison, the artic…

I like to call this Rigor Theater.

Combinatorics: Check

Algorithms: Check

Real world problem: Check

Crowbarred connection between 1-3 to show how your AI algorithm is better? Check

Re: Reconciliation is a knapsack problem

#12

Post by benreesman »

sbolt wrote: post #7
evrimoztamur wrote: post #6

As somebody who worked as an accountant, I saw several times where automated reconciliation solutions devastated books with repeat mistakes and lacking audit logs. I saw interns do the same, too. An automated solution to reconcile statements based on LLM matches removes transparency on how your books are prepared and might create a false sense of trust in the preparation of your books. In case of an audit, people wil…

Sean here, author and EM of the recon team at Modern Treasury. I completely agree, we us AI to surface suggestions to users during manual review so there's full transparency and human confirmation at every step. Our automatic recon doesn't use LLMs for the reasons you outlined.

I’ve made a couple of critical comments and I hope I wasn’t too rude.

The way you present the case in this comment is vastly better than in the post.

There are ambiguous cases in certain accounting treatments, and hinting a resolution is, well it still makes me nervous but it’s not crazy.

Re: Reconciliation is a knapsack problem

#13

Post by sbolt »

benreesman wrote: post #12
sbolt wrote: post #7

Earlier quoted context omitted.

Sean here, author and EM of the recon team at Modern Treasury. I completely agree, we us AI to surface suggestions to users during manual review so there's full transparency and human confirmation at every step. Our automatic recon doesn't use LLMs for the reasons you outlined.

I’ve made a couple of critical comments and I hope I wasn’t too rude. The way you present the case in this comment is vastly better than in the post. There are ambiguous cases in certain accounting treatments, and hinting a resolution is, well it still makes me nervous but it’s not crazy.

All good, and thank you for the feedback!

Re: Reconciliation is a knapsack problem

#14

Post by DecoPerson »

benreesman wrote: post #8

I profoundly disagree that ML models are a good fit for transaction reconciliation. It’s at least arguable that this task is the oldest documented use of writing, and from double-entry accounting to price/time precedence in modern market microstructure, we have algorithms that align very well with human intuition. I can think of few cases where gratuitous application of even simple statistical methods would cause mor…

I have 15 debtors, a few dozen creditors, and 5 employee credit cards. There’s enough transactions that for the 3 years I’ve been reconciling the accounts I have wanted to either write an if/elseif/else-based reconciler assistant, or hire someone, or pay my accountant to do the job.

A few weeks ago, I decided, what the hell, and I spent two days writing a ChatGPT-powered reconciler assistant.

It’s so damn accurate. By feeding it relevant examples, it suggests the right journal entry for each bank transaction nearly every time, including saying “no matching entry” for when the corresponding journal entry hasn’t been posted yet.

It would have taken me a lot longer to write an if/elseif/else-based reconciler, and it have required a lot of manual attention… and the constant internal debate of whether the rules are code that should go in Git or data that should go in the DB.

I think ML models are a great fit for transaction reconciliation because they give good-enough results really fast at a reasonable price. I’d prefer that over continuing to spend my own time, or having to learn the more advanced algorithms you mentioned.

Something is better than nothing.

Re: Reconciliation is a knapsack problem

#15

Post by leejo »

I'm head of processing development at PayProp[^1] where we've been automatically reconciling rental payments for two decades using the techniques described in the article - we just don't call it an AI or a LLM. Our tech saves letting agents huge amounts of time.

We look at the data we have and if it's sufficient we can "automatically" reconcile it - i.e. suggest a match with 100% certainty that the user(s) can then confirm. Otherwise we make an informed suggestions based on all of the likely data from the transaction(s) and sometimes the suggestions are a list of possible matches.

IME the biggest problems in recon are the edge cases around failures in the banking system or the flow that are very difficult to code around and require manual intervention:

* failures of payments X days after they have been reconciled, now you have to pull things apart again * bank reverses transactions but then puts them back and this appears in their intra day statements (MT942 files for example) but doesn't show on their online portals, leading to "duplicates" in one system that aren't really * statement and reference data is incomplete or just wrong (who knew that free text fields can be problematic?) * amounts simply don't match because you invoiced for X and were paid Y - payments are split up to get around constraints, amounts are rounded up, etc.

We deal with these every single day, and we are automating what we can - but you're always going to need a human to confirm the final step in these cases. Perhaps an LLM can improve suggestions, but when the data is just wrong or missing then I'm not so sure.

[^1]: https://us.payprop.com/

Re: Reconciliation is a knapsack problem

#16

Post by thfuran »

DecoPerson wrote: post #14
benreesman wrote: post #8

I profoundly disagree that ML models are a good fit for transaction reconciliation. It’s at least arguable that this task is the oldest documented use of writing, and from double-entry accounting to price/time precedence in modern market microstructure, we have algorithms that align very well with human intuition. I can think of few cases where gratuitous application of even simple statistical methods would cause mor…

I have 15 debtors, a few dozen creditors, and 5 employee credit cards. There’s enough transactions that for the 3 years I’ve been reconciling the accounts I have wanted to either write an if/elseif/else-based reconciler assistant, or hire someone, or pay my accountant to do the job. A few weeks ago, I decided, what the hell, and I spent two days writing a ChatGPT-powered reconciler assistant. It’s so damn accurate. B…

Whether something is better than nothing depends on its error rate and the consequences of an error.

Re: Reconciliation is a knapsack problem

#17

Post by waynesonfire »

whatever1 wrote: post #4

lol at ai for solving deterministic knapsacks. Just get yourself a solver. And if you wanna solve billion item sized problems hire an OR scientist to write a decomposed algo. Literally after 2 minutes of search: https://arxiv.org/pdf/2002.00352.pdf

You searching arxiv.org stood out to me.

I wish I had better skills at searching academic papers for problems I'm trying to solve or that I'm just thinking about. I think just as there are some people that google better than others, I imagine a similar skill applies to academic papers. Anyone encounter this? How do I get better at it?

I thought at first it was an accessibility problem, and perhaps it still is. In that, I didn't have access to a library of academic papers. But, arxiv.org does make available a lot of content for free. The content seems to be growing too.

Another question I'm exploring is how do I decide which journals to subscribe to. I have a limited budget so have to pick wisely. What makes things difficult is that the papers that I have found interesting in the past, seemingly in a related field, are still published to various journals.

One more random comment. I really can't wait until LLMs are applied towards academic papers. Academic papers build on-top of each other and there are concepts that are considered "common knowledge" to experts and may require a long history of papers to consume to build a foundation of concepts and vocabulary. The difficulty is that recursively these papers introduce the same problem. A lot of times the concepts are not that difficult and it would be wonderful if an LLM could be used to fill the gaps as if I were talking to a expert.

I guess there are sort of expository papers that act as a checkpoint for a particular topic. I'm not sure how to find these.

Post reply on HN