GLM 5.2 is nearly as accurate as a human book keeper

>has issues (according to OpenAI)

What is this referring to?

I've been testing various big and small models for years and, about a year ago, switched my attitude from "biggest model is best!" to "best depends on task".

For example, I had a simple coding task which required making 3 trivial changes in 3 source files. Biggest Model completed task perfectly and took 90 seconds.

Smaller Brother also completed task perfectly, but took 30 seconds and cost 5x less.

If and when a large number of companies blindly turn over their accounts payable workflow to some AI agent system, it'll be very interesting to see the "social engineer the LLM" methods that fraud people use to get money sent to them. Basically the same idea as the ancient "send a fax with a bill for an unsolicited delivery of copier toner to 30,000 businesses" but taken into the modern era.

edit: There's already a number of LLM which are intended for outgoing data loss protection to redact or prevent PII from escaping. Is anyone specifically working on a training set and agent that is specialized in reviewing "is this legit to pay", as a sub-task or filtering step in an AP workflow? I suppose it's a GIGO problem, as it would work best only if you have suppliers enrolled in some kind of existing db, with a specific contracted format for invoices, and correlating with project numbers/cost codes.

That "nearly" is doing an awful lot of heavy lifting. It doesn't matter if your AI model is 99% or 99.99% accurate. For a tax return it has to be perfect every time or someone is at best getting a fine or at worst going to prison.

Sure, human error happens too, but humans take accountability. That's why accountants are a regulated profession. Until an AI company CEO is willing to go prison if the output of their model is wrong, these tools are worthless.

But don't take my word for it, head on over to Toot's own terms of service https://toot-books.pages.dev/terms#ai-not-advice

    Toot uses automated and AI systems to generate classifications and reconciliation suggestions. Output may be incomplete or wrong and must be reviewed by you.
    Toot is a software tool. It does not provide accounting, tax, legal, audit, or financial advice, and nothing it produces is a substitute for a qualified accountant or tax adviser. You are responsible for checking Output before approving it or relying on it, and for any decision you make based on it. To the extent permitted by law, we are not responsible for outcomes arising from automated Output you approve without review.

Comparing this to a human book keeper is farcical.

Hahaha non-deterministic accounting probably won’t fly well with the IRS

> This is a prime example of a problem space where accuracy matters, but it also matters who ultimately goes to prison.

I don't disagree that tax fraud is bad but accuracy really doesn't matter 1/10th what people think it does.

Something to keep in mind in the EU with VAT is that there are approximations, errors and then huge frauds like the infamous "carousel fraud". Sadly the carousel fraud is all too common in the EU: this one gets people really sent to jail. But it's not an error a LLM shall make that could lead to a carousel fraud taking place: it's an elaborate scheme.

Now... If you're self-employed and have to pay the VAT (which may or may not be the case depending of the type of self-employment work you do) or if you're a SME and you've got approximations and errors well it's really not a big deal.

Because the public servants in charge of collecting the VAT and verifying that you filled your stuff correctly... Do make shitload of errors too. And then, depending on the country, these public servants have a special privilege: they can decide an amount of VAT+fine (if there's a fine) that you agree with and if you agree with it, their number is the number.

It's everything but a correct number. It's a number they ended up on that is "close enough" (for some definition of "close"). Sometimes by hammering number after number on a little calculator and then ending up saying:

"We ask you to pay 3500 EUR of additional VAT. If you sign here and accept, the IRS (equivalent of the IRS) won't be able to bother you ever again for that year/years".

This notion that, somehow, there's "one" correct number and that public servants paid something like 2 K EUR net per month (which may be the reason why so many are so prompt to ask for bribes btw) can determine it in a few minutes is ridiculous.

It's simply not happening. Usually the accountant doesn't know the exact number. The SME owner doesn't know the real amount of VAT due. And the public servant(s) in charge of a VAT audit, if any takes place, certainly as zero clue as to what the exact VAT number should be.

The rules are way too complicated, there are way too many special cases, and some things are simply extremely hard to take into account (for example during one audit the person told me he disagree with the 90% of 21% VAT deducted for my ISDN line --yup it was a long time ago-- but then I told him it was during a previous TVA audit, a few years before, that the VAT auditor, seen my job, fixed at 90%).

For some stuff you can opt for a fixed number: say a car, there are rules, depending on the country, where you can, say, get back 75% of 50% of the VAT. But people can also, depending on the country, opt-in for the "real usage": you do 12 387 km during the years, how many were really for work? 6 287 km ? Or may 6 275 km?

What if one day while going to work, you took a detour because you want to drop a gift for your grandma's birthday? How were these kilometers accounted for?

Restaurant bills? What if one of the person wasn't work related (your nephew happened to be in town and you had to take car of him and you knew the clients well so they didn't mind: how does the accountant account for that)?

Gifts to client. You got a discount buying 6 packs of bottle of champagne, but not five (no discount if only five), so you bought a sixth one. But that sixth one you kept for you.

Nobody computes those amounts correctly. Nobody.

And don't get me started on the public servants who'll give, during an audit, an unfair advantage to people from their community (whatever that community may be) or those, literally, banging underneath the table while talking about their next vacations: they're telling you an "arrangement" can be made. Basically you give that one person some money (cash bills of course) and, boom, the 20 K EUR of VAT you had to pay suddenly gets lowered down to 5 K EUR.

You refuse? It'd be too bad if those 20 K became 25 K EUR wouldn't it? Mafia-style.

This is all happening. Don't tell me it's not happening because I know people on both sides of these "trades".

Depending on the EU country, it can even be the norm.

As rational people who like determinism (at least I do), at first glance it looks like a SME's accounting is something square but it really ain't. Or let's put it this way: it there's one number that is the really the amount of VAT the SME has to legally pay, I'd guesstimate 0.01% of the SMEs out there pay, to the cent, the proper amount.

All this to say: the approximations, hallucinations and errors of LLMs in accounting for VAT are probably just on par for the course.

P.S: ah yup... The boss (say major airline company) giving a business card and its PIN code and asks to take good care of the clients: and an employee takes them to strip clubs, they all get drunk like crazy, and explodes the expenses on the business card. Way, way, way more than what is OK to put into expenses to entertain clients. Well guess what: I know for a fact this happens too. And I know for a fact this ain't accounted for properly either.