Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 is nearly as accurate as a human book keeper

toot-books.com

Post reply on HN
Print view View original

61–70 of 131 posts

Re: GLM 5.2 is nearly as accurate as a human book keeper

#61

Post by cucumber3732842 »

ofjcihen wrote: post #48
Diogenesian wrote: post #31

This shouldn't be ignored in the discussion here: The job performed by the humans was broader than what was requested of the model in this benchmark: humans also had to find the relevant invoices (searching through mailboxes, or requesting them from providers) and reason through any circumstances which cannot be inferred from the bank feed and invoices/receipts on their own. In the benchmark these circumstances are p…

Hahaha non-deterministic accounting probably won’t fly well with the IRS

The tax code is so complex that business taxes are already nondeterministic.

Re: GLM 5.2 is nearly as accurate as a human book keeper

#62

Post by BeetleB »

krupan wrote: post #59

"The VAT return prepared by the model was essentially correct: the most important number in the return, which is how much VAT the company was owed by the tax agency, was off by only 7 pence relative to the human-prepared return." I don't know how taxes work in Europe, but in the US being "essentially" correct is not good enough for the IRS. The paragraph after this one goes on to explain other mistakes the LLM made?…

> I don't know how taxes work in Europe, but in the US being "essentially" correct is not good enough for the IRS.

It often is. On multiple years they've found errors in my return, and they just fix it for me and bill/refund the difference (usually a refund).

And obviously, the IRS is not going to quibble over 7 pence, given that you round up/down to the nearest dollar anyway.

Re: GLM 5.2 is nearly as accurate as a human book keeper

#65

Post by traverseda »

adamkurkiewicz wrote: post #20
traverseda wrote: post #8

This doesn't surprise me at all. You can really constrain this problem, give very narrow context, and get pretty reliable and reproducible results. I've gotten very good results with some vibe-coded deepseek book keeping. https://github.com/traverseda/beansync Parses emails or other sources, extracts numbers, correlates different transactions, web search, asks questions, stores notes (regex based, very simple). The h…

We've got integrations with major UK banks. Curious if you'd like to use a polished product or be more interested in bank-feed-as-an-API type of use case? What banks do you use?

Credit union atlantic. The possibility of you being able to profitably support small regional credit unions is pretty much 0. Maybe with a general browser use AI and an SMS portal. Also you'd need to bypass anti-bot protections, maybe solve captchas.

In my opinion that is the thing holding back almost all of these accounting products that make consumers lives better. I've solved it because all my cards and bills happen to support email, so I can use that as my source of truth for all my credit cards.

Re: GLM 5.2 is nearly as accurate as a human book keeper

#66

Post by markdown »

krupan wrote: post #59

"The VAT return prepared by the model was essentially correct: the most important number in the return, which is how much VAT the company was owed by the tax agency, was off by only 7 pence relative to the human-prepared return." I don't know how taxes work in Europe, but in the US being "essentially" correct is not good enough for the IRS. The paragraph after this one goes on to explain other mistakes the LLM made?…

The accounting concept of materiality doesn't exist in the US? What about petty cash?

Re: GLM 5.2 is nearly as accurate as a human book keeper

#67

Post by OneDeuxTriSeiGo »

benjiro29 wrote: post #57
zerobees wrote: post #26

This is a prime example of a problem space where accuracy matters, but it also matters who ultimately goes to prison. I'm going to go out on a limb and guess it's not the LLM. If you're acting in good faith and your accountant does something crazy or evil, your liability is limited to some extent. You may get a tax bill but you're probably not gonna end up behind bars. But if your LLM decides to do a little bit of ta…

> If you're acting in good faith and your accountant does something crazy or evil, your liability is limited to some extent. From my understanding, you are the person signing off on the paperwork that is submitted to the IRS. There is this cache 22 with taxes. You are responsible, but you outsource it to a accountant. Because you are not knowledgeable about the taxes. But you are expected to be knowledgeable to under…

This is also something where how you interact with the IRS really really depends on your initial interactions.

If the IRS comes to you and says "your taxes are wrong you owe X amount" and you immediately turn around and prepare to fight them then they are going to be merciless with you.

But if they come to you and you turn around and show the auditor your methodology and what you thought you were supposed to be doing, etc and focus on fixing the issue and settling your debt then they are going to hook you up with IRS staff dedicated to helping fix this kind of stuff.

And most importantly they'll connect you with someone who can set up a payment plan out over whatever period of time to pay back what you owe without killing your business or taking your assets.

But again if you get audited and immediately become adversarial then they'll assume you were doing it intentionally and will take anything not bolted down.

-----

The rationale is basically that the IRS staff are mostly there to help you get your taxes in order and ensure that those taxes are paid but as soon as it starts to look like you are intentionally avoiding paying your taxes you get kicked over to the enforcement divisions and eventually the criminal divisions.

So good faith is absolutely something the IRS will honor but that stops the moment it looks like you are trying to hide stuff from them.

Re: GLM 5.2 is nearly as accurate as a human book keeper

#68

Post by markdown »

adamkurkiewicz wrote: post #27
Obscurity4340 wrote: post #21

Earlier quoted context omitted.

The real test to see if AI is just a rich person thing will be to see how the tax authorities treat it, even for more complex returns. They can save humans for the really complex edge case stuff but at the end of the day, the tax code is just checkboxes and input forms that get boiled down into Integers, Floats/Doubles and enumerated choices with some Strings for deductions

I've submitted my German taxes this year using a mix of Claude 4.6 and Claude 4.7, with lots of manual checking. The German Finanzamt granted most of the things I listed in the tax return (they send you an official letter by post) -- I did have to appeal for one of the items though (again using Claude, this time 4.8 ). The most important thing I've found is to ask Claude to thoroughly audit the reply (to find all hal…

Have you considered using using two completely different models and comparing their output in order to catch hallucinations?

Re: GLM 5.2 is nearly as accurate as a human book keeper

#69

Post by breadislove »

adamkurkiewicz wrote: post #35
Diogenesian wrote: post #31

This shouldn't be ignored in the discussion here: The job performed by the humans was broader than what was requested of the model in this benchmark: humans also had to find the relevant invoices (searching through mailboxes, or requesting them from providers) and reason through any circumstances which cannot be inferred from the bank feed and invoices/receipts on their own. In the benchmark these circumstances are p…

Hey, the author of the benchmark here. The benchmark data was prepared in April 2026 (when I was manually doing our VAT return with my co-founder). The invoices were indeed found manually. Currently we're using a custom "invoice searcher" built on Kimi 2.6 (in our testing several weeks ago it outperformed Opus 4.7; it was just more persistent). Ultimately, I still verify everything manually after the model is finishe…

adam, i'd like to get in touch and would love to run the benachmark with mixedbread as a search backend. we are doing this right now with a lot of compliance companies. would be very curious how it improves quality/cost e2e
Post reply on HN