Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

KPMG pulls report on AI usage due to apparent hallucinations

techcrunch.com

Post reply on HN
Print view View original

31–37 of 37 posts

Re: KPMG pulls report on AI usage due to apparent hallucinations

#31

Post by e12e »

SpicyLemonZest wrote: post #26
e12e wrote: post #15

Earlier quoted context omitted.

> A human can then verify the ones with under 90% certainty. How about the author actually reads the finished report a couple of times and checks all the references? It really is the lowest bar - even lower maybe than running a spell check.

The hallucinations here ( https://gptzero.me/news/investigations-kpmg/ ) would have passed a cursory reference check. It's easy to see when it's laid out in a table that "BNP Paribas. AI Integration: Transforming Financial Journeys. The Banking Scene, 2025." is a false citation, because the title doesn't quite match and it wrongly attributes BNP Paribas authorship to an article written about BNP Paribas by some rando…

I'm not talking about a reference check by someone other than the author. You'd not put a reference in in the first place, that you hadn't read, since you couldn't formulate the text that relates to the reference?

Ed: thanks for the link - I hadn't seen that yet.

Re: KPMG pulls report on AI usage due to apparent hallucinations

#32

Post by a_bonobo »

simonw wrote: post #12

This isn't the first time this has happened, either. I do not understand how these consultancies - who sell these "reports" for six or seven digit sums - continue to mess this up. It should be excruciatingly embarrassing for them. I guess nobody ever got fired for paying KPMG and friends for an expensive report that supported their priors.

KPGM et al. are used as political ammo to push through internal changes. Those in power rely on consultancies underlying their decisions (painful redundancies, firings, etc.). Acknowledging that the arguments for these painful decisions was hallucinated will lead to many problems for powerful people, so for now it's best to just try and sweep it all under the rug.

Re: KPMG pulls report on AI usage due to apparent hallucinations

#33

Post by NoPicklez »

tjwebbnorfolk wrote: post #22
simonw wrote: post #12

This isn't the first time this has happened, either. I do not understand how these consultancies - who sell these "reports" for six or seven digit sums - continue to mess this up. It should be excruciatingly embarrassing for them. I guess nobody ever got fired for paying KPMG and friends for an expensive report that supported their priors.

These six-figure reports are produced by underpaid kids in their twenties working 18 hours a day. The purpose of paying for these reports is for executives to have someone else to blame when their idea doesn't work. It has nothing to do with the correctness of the content.

I would have to disagree, this report in question sounds more like thought leadership dribble rather than a report commissioned by a client with a scope attached.

The purpose of most reports are absolutely for Assurance to decision makers or management and often times, we disagree with management or provide a view that might not favorable. Which just reflects the realities of what we have identified or tested.

As I said, this seems like thought leadership dribble which absolutely even as someone who has worked in Big 4, I think they're pretty average.

Re: KPMG pulls report on AI usage due to apparent hallucinations

#34

Post by overfeed »

tjwebbnorfolk wrote: post #29
overfeed wrote: post #28

Earlier quoted context omitted.

> These six-figure reports are produced by underpaid kids in their twenties working 18 hours a day. That's accurate, for the first draft. Similar to big legal firms - subsequent versions are signed-off and passed up (and if revisions request, down) the hierarchy, each stratum with its own billing rate(s). Which makes me wonder when the hallucinations got added.

Not where I used to work. Any "sign off" was some director making sure the letterhead looked right.

> Not where I used to work

It can't have been at any of the big 4, because partners aren't skipping 4+ org-chart layers to look at draft documents written by early-career associates. I have no experience with body shops - if that's where you were.

Re: KPMG pulls report on AI usage due to apparent hallucinations

#35

Post by yieldcrv »

simonw wrote: post #12

This isn't the first time this has happened, either. I do not understand how these consultancies - who sell these "reports" for six or seven digit sums - continue to mess this up. It should be excruciatingly embarrassing for them. I guess nobody ever got fired for paying KPMG and friends for an expensive report that supported their priors.

> I guess nobody ever got fired for paying KPMG and friends for an expensive report that supported their priors.

I mean basically. KMPG is a regulatory checkmark in some industries

Re: KPMG pulls report on AI usage due to apparent hallucinations

#36

Post by petre »

a_bonobo wrote: post #32
simonw wrote: post #12

This isn't the first time this has happened, either. I do not understand how these consultancies - who sell these "reports" for six or seven digit sums - continue to mess this up. It should be excruciatingly embarrassing for them. I guess nobody ever got fired for paying KPMG and friends for an expensive report that supported their priors.

KPGM et al. are used as political ammo to push through internal changes. Those in power rely on consultancies underlying their decisions (painful redundancies, firings, etc.). Acknowledging that the arguments for these painful decisions was hallucinated will lead to many problems for powerful people, so for now it's best to just try and sweep it all under the rug.

Isn't gmthat exacly how the EU ia tring to regulate AI? Like don't use it for firing people, credit scores etc.

Re: KPMG pulls report on AI usage due to apparent hallucinations

#37

Post by figassis »

danaris wrote: post #27
e12e wrote: post #15

Earlier quoted context omitted.

> A human can then verify the ones with under 90% certainty. How about the author actually reads the finished report a couple of times and checks all the references? It really is the lowest bar - even lower maybe than running a spell check.

How about the author actually, y'know authors the report?

Even then, AI can still be helpful. My point is if you're going to use it, use it right, and not with the intent to deceive.
Post reply on HN