Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Senior European journalist suspended over AI-generated quotes

theguardian.com

Post reply on HN
Print view View original

51–60 of 83 posts

Re: Senior European journalist suspended over AI-generated quotes

#51

Post by ffsm8 »

andrewflnr wrote: post #45
crop_rotation wrote: post #44

Earlier quoted context omitted.

Yes and the world should be utopia and everyone should be happy and we all wish for world peace and yada yada yada. What you are saying is a vision of ideal world as it should be, but doesn't help anyone understand the real world problems.

You can't seriously compare the problem of world peace with the problem of exercising the most basic level of critical thinking w.r.t. LLM output after it has already proven itself unreliable . That's not a utopian dream, it's a level of prudence on par with not sticking a fork in an electrical socket.

You're seriously overestimating the average person's ability to understand what llms are.

Look at all the influences, streamers, podcasters constantly asking em things and taking it as fact - live.

Isn't the joe Rogan experience like the most watched podcast or something? Every episode I've ever stumbled upon he "fact checks" multiple things via their sponsor which is just an llm provider specialized on news.

People aren't good at statistics. If something is close enough to the truth enough times, and talks authoritively on everything with good English... Guess what, they're gonna trust it.

Re: Senior European journalist suspended over AI-generated quotes

#52

Post by WarmWash »

crop_rotation wrote: post #23

HN is full of people saying ABCD should know better and honestly I thought the same, but when I look at almost all of my friends working in critical domains like as a judge or engineer or lawyer or even doctor, they seem to trust ChatGPT more or less blindly. People get defensive when I point out out to them that ChatGPT will make things up and it is widely know, and some even tell me it is the fault of "tech people"…

Every single person, every one of them, that I have watched google something since AI overviews launched, will instantly reference the AI overview. And that model is some bottom-rung high volume model, not even gemini.

Re: Senior European journalist suspended over AI-generated quotes

#53

Post by raw_anon_1111 »

tobr wrote: post #41

Interesting to note how similar this seems to what happened with Benj Edwards at Ars Technica. AI was used to extract or summarize information, and quotes found in the summary were then used as source material for the final writing and never double checked against the actual source. I’ve run into a similar problem myself - working with a big transcript, I asked an AI to pull out passages that related to a certain top…

This seems like a solved problem. Any RAG interface I design I have links to the original source and passage. Even NotebookLM does this.

Re: Senior European journalist suspended over AI-generated quotes

#54

Post by pessimizer »

crop_rotation wrote: post #23

HN is full of people saying ABCD should know better and honestly I thought the same, but when I look at almost all of my friends working in critical domains like as a judge or engineer or lawyer or even doctor, they seem to trust ChatGPT more or less blindly. People get defensive when I point out out to them that ChatGPT will make things up and it is widely know, and some even tell me it is the fault of "tech people"…

> almost all of my friends working in critical domains like as a judge or engineer or lawyer or even doctor, they seem to trust ChatGPT more or less blindly.

We do not live in a meritocracy, because society has no means to judge merit. We live in a society ruled by people who crammed before the tests, and who wrote the papers to agree with and flatter the teacher. Now they are the teachers (and bosses), and

1) expect to be flattered (and LLMs have been built as the ultimate flatterers),

2) feel that a good, ambitious student (or subordinate) will not question them and their work, but instead learn to conform to it, and

3) are not particularly interested in the quality of their work as such, but rather the acceptance of their work. In certain professions, such as judges, doctors, high-level lawyers and engineers, or politicians, they feel like (with good reason) that they can demand acceptance of their work, and punish those who don't accept it.

This position is what they worked so hard as young people for. They were not working to become the best at their jobs. They were working to get the most secure jobs. The most secure jobs are the ones that bad or lazy work doesn't endanger.

Re: Senior European journalist suspended over AI-generated quotes

#55

Post by mh- »

raw_anon_1111 wrote: post #53
tobr wrote: post #41

Interesting to note how similar this seems to what happened with Benj Edwards at Ars Technica. AI was used to extract or summarize information, and quotes found in the summary were then used as source material for the final writing and never double checked against the actual source. I’ve run into a similar problem myself - working with a big transcript, I asked an AI to pull out passages that related to a certain top…

This seems like a solved problem. Any RAG interface I design I have links to the original source and passage. Even NotebookLM does this.

For the curious, the term of art is Grounding.

e.g.: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/g...

Re: Senior European journalist suspended over AI-generated quotes

#57

Post by skygazer »

Out of curiosity, if you asked for the same text extraction multiple times, each inside fresh contexts, is it likely to fabricate unique quotes each time? And if so, a) might that be a procedure we train humans to do to better understand LLM unreliability, and 2) and instrumentalize the behavior to measure answer overlap with non LLM statistical tools?

Also, quote-presence testing/linking against source would seem to be a trivial layer to build on a chat interface, no LLM required. Just highlight and link the longest common strings.

Re: Senior European journalist suspended over AI-generated quotes

#58

Post by Planktonne »

raw_anon_1111 wrote: post #53
tobr wrote: post #41

Interesting to note how similar this seems to what happened with Benj Edwards at Ars Technica. AI was used to extract or summarize information, and quotes found in the summary were then used as source material for the final writing and never double checked against the actual source. I’ve run into a similar problem myself - working with a big transcript, I asked an AI to pull out passages that related to a certain top…

This seems like a solved problem. Any RAG interface I design I have links to the original source and passage. Even NotebookLM does this.

It was already a solved problem with cmd/ctrl + f.

Re: Senior European journalist suspended over AI-generated quotes

#59

Post by philipov »

andrewflnr wrote: post #45
crop_rotation wrote: post #44

Earlier quoted context omitted.

Yes and the world should be utopia and everyone should be happy and we all wish for world peace and yada yada yada. What you are saying is a vision of ideal world as it should be, but doesn't help anyone understand the real world problems.

You can't seriously compare the problem of world peace with the problem of exercising the most basic level of critical thinking w.r.t. LLM output after it has already proven itself unreliable . That's not a utopian dream, it's a level of prudence on par with not sticking a fork in an electrical socket.

You may demand that of yourself, but for others we must design around the fact that they are stupid. You do not have the power to change their stupidity, only your response to it.

Re: Senior European journalist suspended over AI-generated quotes

#60

Post by jacquesm »

crop_rotation wrote: post #23

HN is full of people saying ABCD should know better and honestly I thought the same, but when I look at almost all of my friends working in critical domains like as a judge or engineer or lawyer or even doctor, they seem to trust ChatGPT more or less blindly. People get defensive when I point out out to them that ChatGPT will make things up and it is widely know, and some even tell me it is the fault of "tech people"…

Yes, this is the problem. You give people something that has an oracular interface they will treat it like an oracle.
Post reply on HN