Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ars Technica fires reporter after AI controversy involving fabricated quotes

futurism.com

Post reply on HN
Print view View original

401–409 of 409 posts

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#401

Post by xpe »

guerython wrote: post #288

[flagged]

What you've described seems like a better process than most, by far.

> This is exactly why every AI citation we publish goes through a blocker.

Who is "we"? Kudos!

> What guard rails have other people put in front of AI-written judgements?

A great question. Some 'classic' responses here relate to (a) inter-notator agreement [1]; (b) debate [2]; (c) decomposition / deconstruction [3]; and lots more... computer science is all well and good, but philosophy has studied these topics for thousands of years! Epistemology, in particular, totally slays.

I would also generalize the question: what guard-rails do we need in front of any and all writing? Presumably there is some generative process behind it, but I assume very little now-a-days. Even before the last several years of transformer-fueled mayhem, my confidence even in "highly intelligent" and college-educated people dropped sharply. Talk is cheap, and some people like to talk more than others -- often the kinds of people that I don't find value in listening to.

[1]: Inter-annotator agreement : http://ron.artstein.org/publications/inter-annotator-preprin...

[2]: AI Safety via Debate : https://arxiv.org/abs/1805.00899

[3]: Fact in fragments: Deconstructing complex claims via LLM-based atomic fact extraction and verification : https://www.sciencedirect.com/science/article/abs/pii/S09574...

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#403

Post by bandrami »

etchalon wrote: post #400
bandrami wrote: post #381

Earlier quoted context omitted.

How is it not "outright fabrication"? Quotes were attributed to a subject that he didn't say. That's fabrication.

Fabrication implies agency in a way that doesn't feel accurate.

If you signed your name to the story you settled the agency question right there

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#404

Post by sbarre »

asadotzler wrote: post #362
sbarre wrote: post #266

Earlier quoted context omitted.

Perhaps we should be attributing the "space-related achievements" to Musk's companies and employees, and not to him directly, or at least not solely?

Your comment makes it pretty clear you've not read him much. He regularly credits SpaceX with specific accomplishments and rarely brings Musk into the topic unless it's about setting direction, etc.

I was specifically responding to the person above me, not commenting on the reporter.

> you really can't argue with his space-related achievements

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#405

Post by vasco »

lukan wrote: post #148
vasco wrote: post #147

Earlier quoted context omitted.

The 3 people that work at Reuters actually doing journalism are not doing in ANY way a similar job to the millions writing blog posts for Ars Technica like publications. The latter is an ad seller indeed. And the majority of publications that are renowned also do little to no journalism. It's as if we called "web devs" that learned JS on udemy and just vibe code, Computer Scientists and treated them as if they publis…

Eric Berger at Ars for instance is someone I consider a journalist. Have you proof, that he systematically neglects truth in favor of ad selling?

If I say "plumbers are not electricians" will you also pick a plumber by name and make claims about him for me to refute? None of the points you ask about were made in the comment you replied to.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#406

Post by starkparker »

lich_king wrote: post #398
starkparker wrote: post #316

Earlier quoted context omitted.

As someone who worked as a newspaper copy editor for the first third of my career, "assume that the author is lying to you" was the entire job. A lapse in that non-hypothetically left me responsible, and legally liable, in situations like this.

As someone coming from a family of editors and plugged into the publishing world, I think it would be really weird if that was your job. It's not an adversarial relationship. Your job is to pressure-test the arguments and the language, not to ask every time if maybe the person submitting the article didn't really write it, or didn't really interview the person they're claiming to have interviewed. > A lapse in that n…

> It's not an adversarial relationship. Your job is to pressure-test the arguments and the language, not to ask every time if maybe the person submitting the article didn't really write it, or didn't really interview the person they're claiming to have interviewed

It doesn't have to be adversarial. The things you're describing as part of the job are the things I did to prove that the reporter was doing their job. So was building a relationship with reporters that shared the load of documenting what's verified and how it's verified, so we could both trust that we were each doing our jobs correctly.

> > A lapse in that non-hypothetically left me responsible, and legally liable, in situations like this.

> It didn't.

Aside from the bald-faced arrogance of telling me what did or didn't happen to me, my lawyer, their lawyer, and my publisher's lawyer all sure didn't agree with you. Fortunately for me, you weren't involved in it.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#407

Post by crazygringo »

vintagedave wrote: post #387
crazygringo wrote: post #370

Earlier quoted context omitted.

Because they're busy doing the rest of their job? They don't have enough time for it, nor is it a good use of their time. That's like asking why the CEO of a 20-person startup isn't reading every line of code for bugs. It's not the best use of their valuable time.

It's their responsibility. It is, therefore, absolutely a good use of their time.

That's not how any management position works, which is what an editor is.

You're responsible for verifying that the output looks sane and that processes are good and appear to be followed.

You can't double-check every tiny detail. That's absurd. At some point, you simply have to rely on the word of your employees, and fire them and do damage control if it turns out they're not following procedures but claiming they are.

You seem to be asking for an impossible level of quality control, with the budgets available.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#408

Post by vor_ »

rob_c wrote: post #219
AnonC wrote: post #50

Journalists and bloggers usually write about others’ mess ups and apologies, dissecting which apologies are authentic and which apologies are non-apologies. In this incident, Aurich Lawson of Ars Technica deleted the original article (which had LLM hallucinated quotes) instead of updating it with the error. He then published a vague non-apology, just like large companies and politicians usually do. And now we learn t…

> It’s sad to see Ars Technica at this level. This was from a journalist _who_is_hired_as_expert_ at knowing of/about tooling that hallucinates (LLM ((AI)) chatbots). Decides to implicitly trust said technology to write a "hit piece" (lets be honest it was). In several territories that would fall under slander and if is untrue is a major journalistic mis-step and career ending faux-pas. Why in any situation would the…

I wouldn't characterize the story as a hit piece; the misquotes didn't distort his position. What happened is the LLM accurately summarized his position, but they weren't actually his words in the quotes.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#409

Post by kolinko »

ModernMech wrote: post #218
kolinko wrote: post #162

Earlier quoted context omitted.

Which version? I just tested mine and it replied with an actual quote

The model hosted at chat.com gave me that. It told me what he said about debugging, but also included that made up quote as well which it attributed to him. But what it should have said was "Kernighan has no famous quotes about naming languages, you're probably thinking of the quote by Knuth about the importance of naming in language design."

Full version or free version? Most conplaints people make, I found, was because they use the free version.
Post reply on HN