Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EY Canada published a cybersecurity report and most citations were hallucinated

gptzero.me

Post reply on HN
Print view View original

141–150 of 156 posts

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#141

Post by bluefirebrand »

DrewADesign wrote: post #64
ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

> The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. Yeah probably not for the same reason I left VFX rather than have a lifetime of completely disregarding my own generative creativity and cleaning up LLM-generated bullshit. Fuck that. Double-fuck creating ‘conte…

> In most other jobs, they automate away the only fun part and leave humans with all of the drudgery.

I don't know about "most other jobs", I feel like this applies strongly to software too. AI is being used to automate the fun stuff

You may view writing code as drudgery, I like doing it

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#142

Post by DrewADesign »

bluefirebrand wrote: post #141
DrewADesign wrote: post #64

Earlier quoted context omitted.

> The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. Yeah probably not for the same reason I left VFX rather than have a lifetime of completely disregarding my own generative creativity and cleaning up LLM-generated bullshit. Fuck that. Double-fuck creating ‘conte…

> In most other jobs, they automate away the only fun part and leave humans with all of the drudgery. I don't know about "most other jobs", I feel like this applies strongly to software too. AI is being used to automate the fun stuff You may view writing code as drudgery, I like doing it

Some people enjoy bookkeeping, too.

I loved solving problems with code but I don’t think it’s controversial to say most people enjoy the novel problems much much more than pulling apart an obtusely designed, non-idiomatic API. The AI is better at the details than the novel problem, so whoever is driving still needs to sort that out. In art, writing, and so-on, it only does the interesting part, but it does it poorly. So for professional use cases, it involves a lot of tedious cleanup of other people’s art that a machine (very economically) amalgamated into a pile of shit, and you’ve no opportunity to do anything interesting. At all.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#144

Post by refurb »

mminer237 wrote: post #52
ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matt…

>As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form.

This is my issue with AI.

In the type of work i do the work needs to be precise down to the context of how individual words are used.

Having AI pump out 20 pages of content but then me having to go through the 20 pages word by word, cross checking references and prior statements is going to take a long time.

Not to mention I didn’t write it, so my brain doesn’t already know what’s been written so it takes several passes to confirm its complete and that it all fits together.

I find it easy to just write it myself use AI for the more menial tasks like logic check, completeness checks, etc.

One task AI was very useful in was where we wanted to understand the gaps in a submission relative to a process requirement document. We didn’t care if the output was 100% complete or perfect, we just wanted a few examples.

Being able to input a couple 300 pages documents and have AI spit out a dozen examples in 30 seconds was a huge time saver.

But that was a non-critical task.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#145

Post by erentz »

ilamont wrote: post #8

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the n…

It looks like when you sum up: the cost to generate information using an LLM + the cost to actually verify the information, the result on average is the same as not using the LLM. That does not mean some times it isn’t faster and cheaper. It just means other times it’s slower and more costly. This along with different people’s tolerance for accuracy explains why we see such diverging experiences with it.

So in order to make it pan out the forces at play are trying to make everyone believe we now have to accept wrong, even dangerous results.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#146

Post by KajMagnus »

galaxyLogic wrote: post #17

I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens? I don't know but I would expect it to be realtively easy for an LLM to detect "hallucinations".

I think the AIs don't have enough information about the problem. There's many things those who wrote the prompts forgot to mention. And some of it maybe is tacit knowledge?

Then, it doesn't matter if you add 1000 frontier models -- they still can't generate a good report.

But yes I suppose you can get rid of hallucinated citations though

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#147

Post by CuriousSkeptic »

le-mark wrote: post #48

The real comedy is seeing this garbage come down from senior management, clumsy prompting, hallucinated garbage that’s all fluff and zero actionable information, zero real informed analysis. “See this analysis of our support issues from jira, we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix anything. I’ve seen t…

> we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix

So a net positive then?

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#148

Post by atom058 »

sourcecodeplz wrote: post #66

Was the title updated? from "ernst & young" to EY Canada. Why?

Ernst & Young officially change their name to just "EY" some years back.

Unfortunately, they forgot to check the name beforehand, and it turned out that "EY!" already existed: a gay porn magazine.

Yes, this really happened.

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#149

Post by jimnotgym »

BLKNSLVR wrote: post #138
jimnotgym wrote: post #29

Earlier quoted context omitted.

The Big Four have become a shadow of their former selves. They have become so risk averse that their advice is already incredibly generic and non-actionable. I think their audit work is in a downwards spiral. Audit has become so competitive that they are struggling to find ways to make it cheaper. They have become slaves to reducing the hours booked, and the rate of those hours. To do this they substitute less experi…

I'd hazard a guess that the 'former selves' is a rose-coloured-glasses, nostalgia-induced feeling. It's just that people are more aware / less trusting these days.

You definitely could chat to your audit partner in the past. Otherwise it is the same old bunch of recent graduates giving out advice that they have zero experience to support

Re: EY Canada published a cybersecurity report and most citations were hallucinated

#150

Post by Alifatisk »

How does such thing even happen? I know for example in Qwen Chat or Perplexity, they produce citations on at the end of each generated sentence. So I can hover my mouse over each citation and see from which website that was scraped from.

Did they just prompt ChatGPT with no web search and copy-pasted it?

Post reply on HN