Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI assistants misrepresent news content 45% of the time

bbc.co.uk

Post reply on HN
Print view View original

21–30 of 306 posts

Re: AI assistants misrepresent news content 45% of the time

#21

Post by walkabout »

n4r9 wrote: post #6
MangoToupe wrote: post #5

Now let's run this experiment against the editorial boards in newsrooms. Obviously, AI isn't an improvement, but people who blindly trust the news have always been credulous rubes. It's just that the alternative is being completely ignorant of the worldviews of everyone around you. Peer-reviewed science is as close as we can get to good consensus and there's a lot of reasons this doesn't work for reporting.

I guess the claim is not that rubes did not used to exist, but rather that technology is increasingly encouraging and streamlining rubism.

I decided about a decade ago that McLuhan was a prophet, and that the “message” of the Internet may not include compatibility with democracy, as it turns out.

Re: AI assistants misrepresent news content 45% of the time

#23

Post by ajsnigrutin »

Considering it's EBU with national media (usually taxpayer paid, or paid by some other mandatory way), it would be more interesting if they focused on what the media is reporting now, with human reporters and misleading and other kinds of false reportings. If the frontpage article said something wrong (either by malice or accident), there should be a frontpage article reporting about their error too.

Optimistically that could be extended "twitter-style" by mandatory basic fact checking and reports when they just copy a statement by some politician or misrepresented science stuff (xkcd 1217, X cures cancer), and add the corrections.

But yeah... in my country, with all the 5G-danger craze, we had TV debates with a PhD in telecommunications on one side, and a "building biologist" on the other, so yeah...

Re: AI assistants misrepresent news content 45% of the time

#24

Post by luckydata »

visarga wrote: post #20

I recently tried to get Gemini to collect fresh news and show them to me, and instead of using search it hallucinated everything wholesale, titles, abstracts and links. Not just once, multiple times. I am kind of afraid of using Gemini now for anything related to web search. Here is a sample: > [1] Google DeepMind and Harvard researchers propose a new method for testing the ‘theory of mind’ of LLMs - Researchers have…

Gemini is notoriously bad at tool calling and it's also widely speculated that 3.0 will put an emphasis on fixing that.

Re: AI assistants misrepresent news content 45% of the time

#25

Post by afavour »

delaminator wrote: post #11

According to PEW that's about the same % that trust the BBC's reporting. https://www.pewresearch.org/journalism/fact-sheet/news-media...

Which is a completely different issue. "Do I trust this news network?" is a subjective opinion. Flat out misstating facts and inventing sources is a much more significant problem.

Re: AI assistants misrepresent news content 45% of the time

#27

Post by sofixa »

delaminator wrote: post #11

According to PEW that's about the same % that trust the BBC's reporting. https://www.pewresearch.org/journalism/fact-sheet/news-media...

You seem to be looking at the wrong chart. Around ~50% each of each politically leaning group use BBC as their primary news source.

However, 79% of Brits trust the BBC as per this chart:

https://legacy.pewresearch.org/wp-content/uploads/sites/2/20...

Re: AI assistants misrepresent news content 45% of the time

#28

Post by falcor84 »

AyyEye wrote: post #15
falcor84 wrote: post #9

> 45% of all AI answers had at least one significant issue. > 31% of responses showed serious sourcing problems – missing, misleading, or incorrect attributions. > 20% contained major accuracy issues, including hallucinated details and outdated information. I'm generally against whataboutism, but here I think we absolutely have to compare it to human-written news reports. Famously, Michael Crichton introduced the "Ge…

Human news isn't a good comparison because this is second order -- LMMs are downstream of human news. It's a game of stochastic telephone. All the human error is carried through with additional hallucinations on top.

But the issue is that the vast majority of "human news" is second order (at best), essentially paraphrasing releases by news agencies like Reuters or Associated Press, or scientific articles, and typically doing a horrible job at it.

Regarding scientific reporting, there's as usual a relevant xkcd ("New Study") [0], and in this case even better, there's a fabulous one from PhD Comics ("Science News Cycle") [1].

[0] https://xkcd.com/1295/

[1] https://phdcomics.com/comics/archive.php?comicid=1174

Re: AI assistants misrepresent news content 45% of the time

#29

Post by delusional »

everdrive wrote: post #22

It's important to bear this in mind whenever you find out that someone uses an LLM to summarize a meeting, email, or other communication you've held. That person is not really getting the message you were conveying.

That's a scary thought to me. They're not just outsourcing their thinking. They are actively sabotaging the only tool in their arsenal that could ever supplant it.

I've felt it myself. Recently I was looking as some documentation without a clear edit history. I thought about feeding it into an AI and having it generate one for me, but didn't because I didn't have the time. To think, if I had done that, it probably would have generated a perfectly acceptable edit history but one that would have obscured what changes were actually made. I wouldn't just lack knowledge (like I do now) I would have obtained anti knowledge.

Re: AI assistants misrepresent news content 45% of the time

#30

Post by delusional »

alcide wrote: post #12

Kagi News has been pretty accurate. Source information is provided along with the summary and key details too. AI summarizes are good for getting a feel of if you want to read an article or not. Even with Kagi News I verify key facts myself.

What if the AI makes an interesting or important article sound like one you don't want to read? You'd never cross check the fact, and you'd never discover how wrong the AI was.
Post reply on HN