Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AccountingBench: Evaluating LLMs on real long-horizon business tasks

accounting.penrose.com

Post reply on HN
Print view View original

151–154 of 154 posts

Re: AccountingBench: Evaluating LLMs on real long-horizon business tasks

#151

Post by zachlatta »

samrus wrote: post #123
zachlatta wrote: post #112

Earlier quoted context omitted.

Most small businesses cannot afford CPAs for everyday tasks. At best a CPA signs off on the annual summaries. Most day to day work is done by bookkeepers who are not CPAs. In my area (Vermont) the going rate for a good CPA is $200/hr. Bookkeepers are $20-30/hr.

Most small businesses also cant afford the risk of current LLMs putting garbage in their books that, in the best case, has to be cleaned up or redone, or, in the worst case, gets the IRS up your ass

Tuned LLMs will become more accurate than bookkeepers for most day-to-day small business transactions. I think you underestimate the amount of errors that normal bookkeepers tend to make.

Re: AccountingBench: Evaluating LLMs on real long-horizon business tasks

#152

Post by hommes-r »

Here I feel that having the right tools is going to prevent this kind of drift from happening.

Double entry bookkeeping to make sure the mutations are sound and solving the floating point error by this simple trick: https://news.ycombinator.com/item?id=21687430.

All major fintech's use these two tricks to prevent them from fundamentally screwing up. Believe me I worked at a really big one ;)

Re: AccountingBench: Evaluating LLMs on real long-horizon business tasks

#153

Post by Zerot »

School-Cotton wrote: post #102
Zerot wrote: post #99

Earlier quoted context omitted.

Which is the bigger version number? Version 9.9 or version 9.11? Which is the bigger dollar amount? $9.9 or $9.11? Periods are not always used for the decimal separator but also as a separator for multiple sets of semi-independent numbers.

I have never seen someone write $9.09 as $9.9. What country is this common in?

None. That is the point. 9.9 can be both bigger or smaller than 9.11 depending on context

Re: AccountingBench: Evaluating LLMs on real long-horizon business tasks

#154

Post by skmurphy »

stillpointlab wrote: post #55
o11c wrote: post #25

Earlier quoted context omitted.

Counterpoint: if you detect a human accountant doing this, you can take action against the human. Computers will never meaningfully take the blame, and unfortunately usually mean not blaming any human either.

> you can take action against the human I think that will depend on a case-by-case. I don't have any recent examples but I recall someone trying to sue one of those strip-mall tax preparation franchises over incorrect filings. My understanding is that the documents that you sign when you enroll in those services are pretty strictly in the favor of the company. I doubt you could ever go after the specific "human" that…

The paid tax preparer also signs the return and is still "in the loop" from a liability perspective. See for example

https://www.irs.gov/payments/tax-preparer-penalties

Post reply on HN