But I guess since EY is a CYA hedge anyway, no one really cares about whether the reports are hallucinations or not. Someone high up spent money on EY, so that they can justify some decision and won't be held responsible that much, when it turns out the decision was shit. All that matters to them is, that it has the appearance of something genuine and then they can base the decision on what they receive from EY, which better be what they already wanted to hear/read anyway.
EY Canada published a cybersecurity report and most citations were hallucinated
51–60 of 156 posts
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#52The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…
With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matter.
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#53The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…
> AI output is not getting vetted by knowledgeable people You mean the people they fired and demoralized? One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts." Now, the LLM is a "built-in context expert," and they don't need to vet the o…
Serious orgs are going to have to figure out the human layer. It will be needed, no matter how 'hallucination-free' the AI tooling gets. AI will still have some spectacularly bad fuck ups or even worse time bombs that get embedded in a system and don't become apparent until months or years later.
A lot of this will be dumped on existing staff with predictable results as they don't have the bandwidth to do it right. I can envision "output compliance" or "AI QA" becoming dedicated positions at many orgs. It's clearly needed.
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#54This sort of thing is a complete embarrassment to a firm like EY, where people are paying them a lot of money for advice. They’ve basically demonstrated that their market leading research is just someone asking questions to ChatGPT. If you ever needed evidence to not buy “advice” from such outfits, this is exhibit one. Hopefully they at least fired the partner that published this steaming pile of AI slop.
The Big Four have become a shadow of their former selves. They have become so risk averse that their advice is already incredibly generic and non-actionable. I think their audit work is in a downwards spiral. Audit has become so competitive that they are struggling to find ways to make it cheaper. They have become slaves to reducing the hours booked, and the rate of those hours. To do this they substitute less experi…
Penny wise, pound foolish? Measure twice cut once?
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#55What a horrible page to navigate
Some people should not be allowed to make a website.
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#56~ A greedy, dishonest and unethical capitalist.
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#57This sort of thing is a complete embarrassment to a firm like EY, where people are paying them a lot of money for advice. They’ve basically demonstrated that their market leading research is just someone asking questions to ChatGPT. If you ever needed evidence to not buy “advice” from such outfits, this is exhibit one. Hopefully they at least fired the partner that published this steaming pile of AI slop.
The Big Four have become a shadow of their former selves. They have become so risk averse that their advice is already incredibly generic and non-actionable. I think their audit work is in a downwards spiral. Audit has become so competitive that they are struggling to find ways to make it cheaper. They have become slaves to reducing the hours booked, and the rate of those hours. To do this they substitute less experi…
"don't let the perfect be the enemy of the good" ?
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#58The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…
As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…
I can’t cite “from scratch” for something outside of my knowledge but I side LLM training or assisted search.
Re: EY Canada published a cybersecurity report and most citations were hallucinated
#59The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…
As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…
I do the second approach for coding with smallish steps and the output is fine