EY Canada published a cybersecurity report and most citations were hallucinated • Hacker News Board

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#101

Post by asdff » Sat, May 30, 2026, 9:16 PM UTC

ilamont wrote: post #53

ChrisMarshallNY wrote: post #31

Earlier quoted context omitted.

> AI output is not getting vetted by knowledgeable people You mean the people they fired and demoralized? One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts." Now, the LLM is a "built-in context expert," and they don't need to vet the o…

> Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore. Serious orgs are going to have to figure out the human layer. It will be needed, no matter how 'hallucination-free' the AI tooling gets. AI will still have some spectacularly bad fuck ups or even worse time bombs that get embedded in a system and don't become apparent until months or years later. A lot of this will be dumped…

Let's be honest, how many orgs are really serious? Playing the game of the day for shareholder appeasement is taken far more seriously than whatever the domain experts might think.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#102

Post by tipsytoad » Sat, May 30, 2026, 9:16 PM UTC

Who designs a website like this?

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#103

Post by smartmic » Sat, May 30, 2026, 9:19 PM UTC

sourcecodeplz wrote: post #66

Was the title updated? from "ernst & young" to EY Canada. Why?

Not by me, but by the mods. They also changed from "full of hallucinations" to "and most citations were hallucinated". Maybe a rep from "EY Global" filed a complain ;)

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#104

Post by 2fff » Sat, May 30, 2026, 9:22 PM UTC

csomar wrote: post #79

mminer237 wrote: post #52

Earlier quoted context omitted.

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…

It's not really any different in programming. Like if you have a well structured code and want to do a clear refactoring across it and you know what to expect, it can speed things up. But if it's generating any significant (and relatively complex) new code, you have to go through the whole thing manually again and then you find out you have to fix way to many things and get bogged down in different paths the AI didn'…

Youre not wrong I believe.

I think its the perfect time to be contrarian - think about it. If youre wrong - So what? The world will have changed for everyone in the field. If you are right? You stand to be positioned to win big financially whilst everyone elses brain is rotting away.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#105

Post by 2fff » Sat, May 30, 2026, 9:26 PM UTC

fragmede wrote: post #92

onlyrealcuzzo wrote: post #63

Earlier quoted context omitted.

The interesting thing is... There may be a lot of demand for do-nothing services. A lot of corporate work is just do-nothing box-ticking. Boss: get me a report about X, so I can give that report to my boss who won't read it. You: E&Y, please get me a report. Here's $200k.

The trope about external consultants is that your VP brings them in to review the company, and they talk to everybody and write a report on how to improve the business, and the report says exactly what you've been telling your VP but they've been ignoring you.

You are closer to the truth :)

they are not simply paid to do nothing. They are paid to do dirty work.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#106

Post by zelphirkalt » Sat, May 30, 2026, 9:33 PM UTC

krapp wrote: post #87

zelphirkalt wrote: post #51

I wish we could just stop destroying people's jobs and lives using AI. The statistics I have heard quoted say, that merely 25% of the people actually like their job. Meaning they like doing what they do for its own sake, not because it gets them money, which they desperately need to live. I get it, most people don't want to do the work. But can we stop ruining the jobs of people, who are actually dedicated to their j…

>The statistics I have heard quoted say, that merely 25% of the people actually like their job. Meaning they like doing what they do for its own sake, not because it gets them money, which they desperately need to live. Even people who like their jobs work because they need money to live.

My point is, that people who want to do their job properly are less likely to sling AI slop and be found out to do that, and that I wish we could stop destroying their jobs or lives, to chase stakeholder wet dreams of the companies they invested in letting go almost everyone.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#107

Post by themafia » Sat, May 30, 2026, 9:38 PM UTC

FearNotDaniel wrote: post #81

themafia wrote: post #75

Title changed to remove "Earnst & Young". Why? It seems deferential to an entity that, in this case, certainly doesn't deserve it.

Probably because (a) that’s not their name any more and (b) when it was, that’s not how you spell it

Which is interesting because a) the site title includes it and b) I obviously made a common mistake which has nothing to do with the point at hand.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#108

Post by tamimio » Sat, May 30, 2026, 9:39 PM UTC

Those are who rejected you for a job you applied for.. AI amplified the dunning kruger that unfortunately real experts in their field are overlooked now, because a wall of text with numbers sounds and look professional enough.

Any person with above average knowledge on a specific topic, can tell when AI starts hallucinating and making things up, or at least introducing new problems due to complexity added rather than solving it, that’s my observation using all top tier ones too, it’s like they are designed to solve a problem regardless so they start making things up or piling workarounds, a person with no deep knowledge in that topic will just copy it all and call it a day.

Just yesterday, I asked claude 4.8 on something specific that I know the answer for, it had a long list of solutions that none were close to the real answer, when I replied with the real answer and pushed back, I got the famous quote “you are right, thanks for pushing back”.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#109

Post by cassianoleal » Sat, May 30, 2026, 9:39 PM UTC

anal_reactor wrote: post #98

ilamont wrote: post #53

Earlier quoted context omitted.

> Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore. Serious orgs are going to have to figure out the human layer. It will be needed, no matter how 'hallucination-free' the AI tooling gets. AI will still have some spectacularly bad fuck ups or even worse time bombs that get embedded in a system and don't become apparent until months or years later. A lot of this will be dumped…

> It's clearly needed. Once the hallucination rate drops below error rate of human workers, it won't be needed anymore.

Once the hallucination rate drops, the remaining LLM failures will become increasingly harder to spot.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#110

Post by claaams » Sat, May 30, 2026, 9:51 PM UTC

mminer237 wrote: post #52

ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…

This is the realization I had too. We had a manager update a policy at our org. He just shit it out through AI. It had tons of mistakes, people who read it had questions. Not only did it have mistakes it was causing people to do things in a way that added a manual step when an automatic process existed. Then the engineer VP commented on it asking the original author what its about who then had to bring it back up to the attention of the manager who made the first change.

It wasted many people's time, probably an order of magnitude of time wasted (and money) than if the initial person put a modicum of effort into making it right in the first place. Instead they hand it off to their life partner claude and just assume its good enough.

It's to the point where I am feeling insulted when I get ai slop like this from people. If I am expected to perform at a high level then I expect that at the very minimum the slop throwers will proof read their slop.