Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EY Canada published a cybersecurity report and most citations were hallucinated

gptzero.me

Post reply on HN
Print view View original

91–100 of 156 posts

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#91

Post by s0rce »

_puk wrote: post #12
ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

Part of the problem: you get given a complete document to review after it's been fully baked. I'm pushing the need for basic engineering principles across whole organisations. You wouldn't give an engineer 1000 lines of code to review without the original spec of what you're trying to achieve for context (at a minimum, ideally the reviewer was in the room when the work was introduced, and has full context). So, these…

I've had this situation and basically just had to throw out stuff that was written because its completely terrible/wrong. Either start again or just give up.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#92

Post by fragmede »

onlyrealcuzzo wrote: post #63
mapontosevenths wrote: post #11

EY has been quietly laying people off for the last year solid. It's unsurprising that trying to do more with less results in lower quality.

The interesting thing is... There may be a lot of demand for do-nothing services. A lot of corporate work is just do-nothing box-ticking. Boss: get me a report about X, so I can give that report to my boss who won't read it. You: E&Y, please get me a report. Here's $200k.

The trope about external consultants is that your VP brings them in to review the company, and they talk to everybody and write a report on how to improve the business, and the report says exactly what you've been telling your VP but they've been ignoring you.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#95

Post by smelendez »

mminer237 wrote: post #52
ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…

Fact-checking and editing a mediocre piece of writing be way harder than writing from scratch. Proving that something isn’t true or can’t be substantiated is hard work, and so is arguing that a word choice is subtly inappropriate.

And making a ton of corrections to a document everyone was hoping was ready to go is never fun politically.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#96

Post by __turbobrew__ »

mminer237 wrote: post #52
ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…

I have experienced this several times lately when writing software with claude/codex. Sometimes vetting and steering the agent takes longer than it would have taken me if done manually. Sure you can just decide not to vet the output and go into full vibecode, but agents tend to do a lot of dumb things (such as not deleting unused private methods or having temporary variables that are not needed).

In my experience the most effective work pattern for me is using agents to perform research and feedback on high level design, then I write the code manually, then I ask the agent to review the code for potential bugs/issues and fix those. The agents have a much easier time making small changes once the design is 90% there without going fully off the rails and generating slop.

I am working on writing skills to make the agent better but it is a bit painstaking. For example I had to write this inside of a skill because sometimes the agent would just stub out methods and leave TODOs: “always fully complete the requested task before finishing edits unless input is needed”.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#97

Post by dragonfax »

FearNotDaniel wrote: post #78

Off topic but: the scroll mechanism on mobile is so horribly irritating and unpredictable that I just can’t be bothered fighting against it to read what sounds like at least a mildly interesting article.

No, it's like that on desktop too.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#98

Post by anal_reactor »

ilamont wrote: post #53
ChrisMarshallNY wrote: post #31

Earlier quoted context omitted.

> AI output is not getting vetted by knowledgeable people You mean the people they fired and demoralized? One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts." Now, the LLM is a "built-in context expert," and they don't need to vet the o…

> Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore. Serious orgs are going to have to figure out the human layer. It will be needed, no matter how 'hallucination-free' the AI tooling gets. AI will still have some spectacularly bad fuck ups or even worse time bombs that get embedded in a system and don't become apparent until months or years later. A lot of this will be dumped…

> It's clearly needed.

Once the hallucination rate drops below error rate of human workers, it won't be needed anymore.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#99

Post by hansmayer »

ilamont wrote: post #53
ChrisMarshallNY wrote: post #31

Earlier quoted context omitted.

> AI output is not getting vetted by knowledgeable people You mean the people they fired and demoralized? One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts." Now, the LLM is a "built-in context expert," and they don't need to vet the o…

> Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore. Serious orgs are going to have to figure out the human layer. It will be needed, no matter how 'hallucination-free' the AI tooling gets. AI will still have some spectacularly bad fuck ups or even worse time bombs that get embedded in a system and don't become apparent until months or years later. A lot of this will be dumped…

[dead]

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#100

Post by VTuberTTV »

__turbobrew__ wrote: post #96
mminer237 wrote: post #52

Earlier quoted context omitted.

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…

I have experienced this several times lately when writing software with claude/codex. Sometimes vetting and steering the agent takes longer than it would have taken me if done manually. Sure you can just decide not to vet the output and go into full vibecode, but agents tend to do a lot of dumb things (such as not deleting unused private methods or having temporary variables that are not needed). In my experience the…

[dead]
Post reply on HN