Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Editor's Note: Retraction of article containing fabricated quotations

arstechnica.com

Post reply on HN
Print view View original

71–80 of 223 posts

Re: Editor's Note: Retraction of article containing fabricated quotations

#71

Post by mrandish »

qnleigh wrote: post #45
mrandish wrote: post #43

When an article is retracted it's standard to at least mention the title and what specific information was incorrect so that anyone who may have read, cited or linked it is informed what information was inaccurate. That's actually the point of a retraction and without it this non-standard retraction has no utility except being a fig leaf for Ars to prevent external reporting becoming a bigger story. In the comments I…

Yes I just read the retracted article and I can't find anything that I knew was false. What were the fabricated quotes?

I was wondering the same thing. After I posted above, I followed the archive.org link to the original article and did a quick search on the last four quotes, which the article claims are from Scott's blog. None appear on the linked blog page. The first quote the article claims is from Scott does appear on the linked Github comments page.

When I wrote my post above, I hadn't yet read the original article on achive.org. Now that I know the article actually links to the claimed original sources on Scott's blog and Github for all the fabricated quotes, how this could have happened is even more puzzling. Now I think this may be much more interesting than just another case of "lazy reporter uses LLM to write article".

Ars appears to use an automated tool which adds text links to articles to increase traffic to any related articles already on Ars. If that tool is now LLM-based to allow auto-generating links based on concepts instead of just keywords, perhaps it mistakenly has unconstrained access to changing other article text! If so, it's possible the author and even the editors may not be at fault. The blame could be on the Ars publisher's using LLM's to automate monetization processes downstream of editorial. Which might explain the non-standard vague retraction. If so, that would make for an even more newsworthy article that's directly within Ars' editorial focus.

Re: Editor's Note: Retraction of article containing fabricated quotations

#72

Post by rstuart4133 »

j0057 wrote: post #9

Odd that there's no link to the retracted article. Thread on Arstechnica forum: https://arstechnica.com/civis/threads/editor%E2%80%99s-note-... The retracted article: https://web.archive.org/web/20260213194851/https://arstechni...

The articles authors are listed on that page as: Benj Edwards and Kyle Orland.

Re: Editor's Note: Retraction of article containing fabricated quotations

#73

Post by mrandish »

apparent wrote: post #61
mrandish wrote: post #43

When an article is retracted it's standard to at least mention the title and what specific information was incorrect so that anyone who may have read, cited or linked it is informed what information was inaccurate. That's actually the point of a retraction and without it this non-standard retraction has no utility except being a fig leaf for Ars to prevent external reporting becoming a bigger story. In the comments I…

In the case of hallucinated quotes, I think the more important aspect is to describe how this happened, whether the author is a regular contributor, how the editors missed it, and what steps are being taken to prevent it from happening in the future. It's good to issue a correction, and in this case to retract the article. But it doesn't really give me confidence going forward, especially where this was flagged becau…

> It's not like Ars' own processes somehow unearthed this error.

It might be even worse (and more interesting) than that. I just posted a sister response outlining why I now suspect the fabrication may have actually been caused by Ars' own process. https://news.ycombinator.com/item?id=47027370. Hence, the odd non-standard retraction.

Re: Editor's Note: Retraction of article containing fabricated quotations

#74

Post by icegreentea2 »

maxbond wrote: post #60
icegreentea2 wrote: post #56

Earlier quoted context omitted.

Yeah, but the problem is that by not making it clear that additional actions may be coming, they're barely restoring credibility at all, because the current course of action (pulling the article and saying sorry) is like the bare minimal required to avoid being outright liars - a far cry from being credible journalists. All they've done is leave piles of readers (including Ars subscribers) going "wtf". If they felt t…

You don't announce that you're firing people or putting them on a PIP or something. Not only is it gauche but it makes it seem like you're not taking any accountability and putting it all in the employees involved. I assume their AI policy is fine and that the issue was it wasn't implemented/enforced, and I'm not sure what they can do about that other than discipline the people involved and reiterate the policy to ev…

They just needed to expand "At this time, this appears to be an isolated incident." into "We are still investigating, however at this time, this appears to be an isolated incident". No additional details required.

And yes, it looks like Ars is still investigating (bluesky post by one of the authors of the retracted article) https://bsky.app/profile/kyleor.land/post/3mewdlloe7s2j

Re: Editor's Note: Retraction of article containing fabricated quotations

#76

Post by knowitnone3 »

arduanika wrote: post #35
anonymous908213 wrote: post #8

Earlier quoted context omitted.

> Greatly appreciate this direct statement clarifying your standards, and yet another reason that I hope Ars can remain a strong example of quality journalism in a world where that is becoming hard to find > Kudos to ARS for catching this and very publicly stating it. > Thank you for upholding your journalistic standards. And a note to our current administration in DC - this is what transparency looks like. > Thank y…

No reason to trust that the comment section is any more genuine than the deleted fake article. If an Ars employee used genAI to astroturf these comments, they clearly would not be fired for it or even called out by name.

[dead]

Re: Editor's Note: Retraction of article containing fabricated quotations

#77

Post by ezst »

delichon wrote: post #64

Imagine a future news environment where oodles of different models are applied to fact check most stories from most major sources. The markup from each one is aggregated and viewable. A lot of the results would be predictable partisan takes and add no value. But in a case like this where the whole conversation is public, the inclusion of fabricated quotes would become evident. Certain classes of errors would become l…

So, the solution to too much AI is... Even more AI! You sound like you would fit just right at a LLM-shop marketing department.

Re: Editor's Note: Retraction of article containing fabricated quotations

#78

Post by anonymous908213 »

maxbond wrote: post #67
anonymous908213 wrote: post #59

Earlier quoted context omitted.

I don't believe there is any greater journalistic malpractice than fabrication. Sure, there are worse cases of such malpractice in the world given the low importance of the topic, but journalists should be reporting the truth on anything they deem important enough to write about. Cutting corners on the truth, of all things, is the greatest dereliction of their duty, and undermines trust in journalism altogether, whic…

> I don't believe there is any greater journalistic malpractice than fabrication. Sure, there are worse cases of such malpractice... That's more or less what I mean. It was only a few notches above listicle to begin with. I don't think they intended to fabricate quotes. I think they didn't take the necessary time because it's a low-stakes, low-quality article to begin with. With a short shelf life, so it's only valua…

What I'm saying is that I believe they do not care about the truth, and intentionally chose to offload their work to LLMs, knowing that LLMs do not produce truth, because it does not matter to them. Is there any indication that this has damaged their career in any way? It seems to me that it's likely they do not care about the truth because Ars Technica does not care about the truth, as long as the disregard isn't so blatant that it causes a PR issue.

> I also feel compelled to point out you've abandoned your claim that the article was generated.

As you've pointed out, neither of us has a crystal ball, and I can't definitively prove the extent of their usage. However, why would I have any reason to believe their LLM usage stops merely at fabricating quotes? I think you are again engaging in the most charitable position possible, for things that I think are probably 98 or 99% likely to be the result of malicious intent. It seems overwhelmingly likely to me that someone who prompts an LLM to source their "facts" would also prompt an LLM to write for them - it doesn't really make sense to be opposed to using an LLM to write on your behalf but not be opposed to it sourcing stories on your behalf. All the more so if your rationale as the author is that the story is unimportant, beneath you, and not worth the time to research.

Re: Editor's Note: Retraction of article containing fabricated quotations

#79

Post by anonymous908213 »

delichon wrote: post #64

Imagine a future news environment where oodles of different models are applied to fact check most stories from most major sources. The markup from each one is aggregated and viewable. A lot of the results would be predictable partisan takes and add no value. But in a case like this where the whole conversation is public, the inclusion of fabricated quotes would become evident. Certain classes of errors would become l…

The author of the blog post hypothesised that the fabrication happened as a result of measures blocking LLMs from scraping their blog. If that is the case, adding more LLMs would not in fact accomplish anything at all.

Re: Editor's Note: Retraction of article containing fabricated quotations

#80

Post by maxbond »

anonymous908213 wrote: post #78
maxbond wrote: post #67

Earlier quoted context omitted.

> I don't believe there is any greater journalistic malpractice than fabrication. Sure, there are worse cases of such malpractice... That's more or less what I mean. It was only a few notches above listicle to begin with. I don't think they intended to fabricate quotes. I think they didn't take the necessary time because it's a low-stakes, low-quality article to begin with. With a short shelf life, so it's only valua…

What I'm saying is that I believe they do not care about the truth, and intentionally chose to offload their work to LLMs, knowing that LLMs do not produce truth, because it does not matter to them. Is there any indication that this has damaged their career in any way? It seems to me that it's likely they do not care about the truth because Ars Technica does not care about the truth, as long as the disregard isn't so…

> I think you are again engaging in the most charitable position possible, ...

Yeah, that's accurate. I will turn a dime the moment I receive evidence that this was routine for this author or systemic for Ars. But yes, I'm assuming good faith (especially on Ars' part), and that's generally how I operate. I guess I'm an optimist, and I guess I can't ask you to be one.

Post reply on HN