GLM 5.2 is nearly as accurate as a human book keeper
toot-books.com
GLM 5.2 is nearly as accurate as a human book keeper
1–10 of 131 posts
Re: GLM 5.2 is nearly as accurate as a human book keeper
#2> nearly as accurate as a human book keeper
Anything to avoid using the metric system.
Though seriously, what is this metric? Why would I care if an LLM is accurate as a human bookkeeper? Humans aren't exactly known for perfect recall.
Re: GLM 5.2 is nearly as accurate as a human book keeper
#3> nearly as accurate as a human book keeper Anything to avoid using the metric system. Though seriously, what is this metric? Why would I care if an LLM is accurate as a human bookkeeper? Humans aren't exactly known for perfect recall.
You'd care if you had a human bookkeeper and you were considering replacing them with this company's AI bookkeeper.
Re: GLM 5.2 is nearly as accurate as a human book keeper
#4> nearly as accurate as a human book keeper Anything to avoid using the metric system. Though seriously, what is this metric? Why would I care if an LLM is accurate as a human bookkeeper? Humans aren't exactly known for perfect recall.
You'd care if you had a human bookkeeper and you were considering replacing them with this company's AI bookkeeper.
Why would I want both a less accurate book keeper and to incur all of the liability of doing the books myself?!
Re: GLM 5.2 is nearly as accurate as a human book keeper
#5> They've done studies, you know. Sixty percent of the time, it works every time.
Re: GLM 5.2 is nearly as accurate as a human book keeper
#6Interesting write-up. Having been a bookkeeper a long time ago, I'm not too surprised at this being susceptible to automation by an LLM backed system.
It seems also that the classes of error they encountered could be handled by improved skills/knowledge base access on the fine points of relevant tax legislation.
The important part for their software ofc is, will they take responsibility for the output if HMRC come calling? Without that users are adopting the risk which they may not be keen to do (dealing with HMRC is not fun), with that it could be a very nice saving for a lot of small companies (and bad for the employees of a lot of accountancy firms)
Re: GLM 5.2 is nearly as accurate as a human book keeper
#7Earlier quoted context omitted.
You'd care if you had a human bookkeeper and you were considering replacing them with this company's AI bookkeeper.
Why would I want both a less accurate book keeper and to incur all of the liability of doing the books myself?!
Presumably your current book keeper is not your slave, and you have to pay them...
Re: GLM 5.2 is nearly as accurate as a human book keeper
#8This doesn't surprise me at all. You can really constrain this problem, give very narrow context, and get pretty reliable and reproducible results.
I've gotten very good results with some vibe-coded deepseek book keeping. https://github.com/traverseda/beansync
Parses emails or other sources, extracts numbers, correlates different transactions, web search, asks questions, stores notes (regex based, very simple).
The hard part is getting good data, I'm sure that lexus nexus or whoever can get API access to my bank account and all my credit cards, but I can't. Email turned out to be the best way for most of my providers. Managed to avoid 2factor auth so far, but it will suck when I need it.
Re: GLM 5.2 is nearly as accurate as a human book keeper
#9Oh, I'm actively doing this at the moment. FreeAgent grabs my transactions from Wise already, and then I give it [Claude Code, in fact] a folder of PDFs to attach to my invoices, including figuring out VAT, and it's uploading what it found using the FreeAgent API. My accountant hasn't complained yet, and it seems considerably more accurate than when my wife was doing it.
Quiet plug for https://github.com/pjlsergeant/byre which I use for all my little projects like this.
Re: GLM 5.2 is nearly as accurate as a human book keeper
#10> nearly as accurate as a human book keeper Anything to avoid using the metric system. Though seriously, what is this metric? Why would I care if an LLM is accurate as a human bookkeeper? Humans aren't exactly known for perfect recall.
Hey, author of the blog post here.
I was one of the human book-keepers for this benchmark (the preparer; my co-founder verified the VAT submission once ready), and given that at the time of doing this I knew I was eventually going to use this data for evaluating the models, I was super careful. So I guess this is a "good book-keeper". In the previous company our book-keepers made lots of mistakes; some serious enough that we had to restate our company's accounts.