Large Language Models (LLMs), lacking true comprehension of the underlying concepts, convert sequences of text into numerical vectors known as tokens. Using a prediction engine together with user input, attempt to predict the next token in the sequence. As such - it's all hallucinations.
AI assistants misrepresent news content 45% of the time
261–270 of 306 posts
Re: AI assistants misrepresent news content 45% of the time
#262Earlier quoted context omitted.
The fix for this is for the AI to double-check all links before providing them to the user. I frequently ask ChatGPT to double check that references actually exist when it gives me them. It should be built in!
But that would mean OpenAI would lose even more money on every query.
Better make each query count then.
Re: AI assistants misrepresent news content 45% of the time
#263Earlier quoted context omitted.
We live in a post-truth society. This means that, unfortunately, most of society has learned that it doesn't matter if what you're saying is true. All that matters is that the words that you speak cause you or your cause to gain power.
All the more reason to call out bullshit in real life Value truth and honesty. Call out lies for what they are. This is the way to get sanity both for ourselves and for society as a whole
[dead]
Re: AI assistants misrepresent news content 45% of the time
#264Earlier quoted context omitted.
> or it (shocking) cites Wikipedia instead of the BBC. No... the problem is that it cites Wikipedia articles that don't exist . > ChatGPT linked to a non-existent Wikipedia article on the “European Union Enlargement Goals for 2040”. In fact, there is no official EU policy under that name. The response hallucinates a URL but also, indirectly, an EU goal and policy.
Do we have any good research on how much less often larger, newer models will just make stuff up like this? As it is, it's pretty clear LLMs are categorically not a good idea for directly querying for information in any non-fiction-writing context. If you're using an LLM to research something that needs to be accurate, the LLM needs to be doing a tool call to a web search and only asked to summarize relevant facts fr…
> and only asked to summarize relevant facts from the existing information it can find
Still not enough as I find the LLM will not summarize all the relevant facts, sometimes leaving out the most salient ones. Maybe you'll get a summary of some facts, maybe the ones you explicitly ask for, but you'll be left wondering if the LLM is leaving out important information.
Re: AI assistants misrepresent news content 45% of the time
#265Earlier quoted context omitted.
A further problem is that Wikipedia is chock full of nonsense, with a large proportion of articles that were never fact checked by an expert, and many that were written to promote various biased points of view, inadvertently uncritically repeat claims from slanted sources, or mischaracterize claims made in good sources. Many if not most articles have poor choice of emphasis of subtopics, omit important basic topics,…
Wikipedia is pretty good for most topics. Anything even remotely political somewhere however, it isn't just bad, it is one of the worst sources out there. And therein lies the problem, its wildly different levels of quality depending on the topic.
Wikipedia is bad even for topics that aren't particularly political, not even because the editor was trying to be misleading but rather was being lazy and wrote up their own misconception and either made up a source or pulled a source without bothering to actually read it. These kind of errors can stay in place for years.
I have one example that I check periodically just to see if anybody else has noticed. I've been checking it for several years and it's still there; the SDI page claims that Brilliant Pebbles was designed to use "watermelon sized" tungsten projectiles. This is completely made up; whoever wrote it up was probably confusing "rods from god" proposals that commonly use tungsten and synthesizing that confusion with "pebbles". The sentence is cited but the sources don't back it up. It's been up like this for years. This error has been repeated on many websites now, all post-dating the change on wikipedia.
If you're reading this and are the sort to edit wikipedia.. Don't fix it. That would be cheating.
Re: AI assistants misrepresent news content 45% of the time
#266Earlier quoted context omitted.
Human journalists misrepresent the white paper 85% of the time. With this in mind, 45% doesn't seem so bad anymore
Years ago in college, we had a class where we analyzed science in the news for a few weeks compared to the publish research itself. I think it was a 100% misrepresentation rate comparing what a news article summarized about a paper verses what the paper itself said. We weren't going off of CNN or similar main news sites, but news websites aimed at specific types of news which were consistently better than the article…
That's terrific. Media literacy should be required civics curriculum.
I was on my highschool's radio station, part of the broadcast media curriculum. It was awesome.
That early experience erased any esteem I had for mass media. (Much as I loved the actual work.)
We got to visit local stations, job shadow, produce content for public access cable, make commercials, etc. Meet and interview adults.
We also talked with former students, who managed to break into the industry.
Since it was a voc tech program, there was no mention of McLuhan, Chomsky, Postman, or any media criticism of any kind.
I learned that stuff much later. Yet somehow I was able to intuit the rotten core of our media hellscape.
Re: AI assistants misrepresent news content 45% of the time
#267I'm curious how many people have actually taken the time to compare AI summaries with sources they summarize. I did for a few and ... it was really bad. In my experience, they don't summarize at all, they do a random condensation.. not the same thing at all. In one instance I looked at the result was a key takeaway being the opposite of what it should have been. I don't trust them at all now.
Sometimes they do random fabrication. I saw one AI cite a paper that didn't exist. Fictitious title, authors, and results.
Re: AI assistants misrepresent news content 45% of the time
#268Earlier quoted context omitted.
They can be good for search, but you must click through the provided links and verify that they actually say what it says they do.
The problem is that 90% of people will not do that once they've satisfied their confirmation bias. Hard to say if that's going to be better or worse than the current echo chamber effects of the Internet. I'm still holding out for better, but certainly this is shaking that assumption
So this probably is valid. However, so is Gell-Mann amnesia and both phenomena happen a lot. There are topics where one side is the group of people who have attempted to understand a problem and the other side are people who either do not or won't due to emotions. Acting as if it is all confirmation bias feels good but probably isn't the best way to look at the media.
Re: AI assistants misrepresent news content 45% of the time
#269Kagi News has been pretty accurate. Source information is provided along with the summary and key details too. AI summarizes are good for getting a feel of if you want to read an article or not. Even with Kagi News I verify key facts myself.
How do you verify a fact? Do you travel to the location and interview the locals? Or read scientific papers in various fields, including their own references, to validate summaries published by news sources? At some point you need to just trust that someone is telling the truth.
It may help to set penalties for someone not telling the truth.
Re: AI assistants misrepresent news content 45% of the time
#270I'm curious how many people have actually taken the time to compare AI summaries with sources they summarize. I did for a few and ... it was really bad. In my experience, they don't summarize at all, they do a random condensation.. not the same thing at all. In one instance I looked at the result was a key takeaway being the opposite of what it should have been. I don't trust them at all now.
They’re basically markov chain text generators with a relevance-tracking-and-correction step. It turns out this is like 100x more useful than the same thing without the correction step, but they don’t really escape what they are “at heart”, if you will. The ways they fail are often surprising if your baseline is “these are thinking machines”. If your baseline is what I wrote above (say, because you read the “Attentio…
See also: 3 Blue 1 Brown's fantastic series on deep learning, particularly videos like "Transformers, the tech behind LLMs".
My own mental model (condensed to a single phrase) is that LLMs are extremely convincing (on the surface) autocomplete. So far, this model has not disappointed me.