Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our newsroom AI policy

arstechnica.com

Post reply on HN
Print view View original

31–40 of 144 posts

Re: Our newsroom AI policy

#31

Post by klausa »

applfanboysbgon wrote: post #13
brey wrote: post #11

Earlier quoted context omitted.

The next sentence after your quoted section: “Even then, AI output is never treated as an authoritative source. Everything must be verified.”

Any verification process thorough enough to catch all LLM fabrications would take more work than simply not using the LLM in the first place. If anything verifying what an LLM wrote is substantially more difficult than just reading the material it's "summarising", because you need to fully read and comprehend the material and then also keep in mind what the LLM generated to contrast and at that point what the fuck ar…

The LLM can find material that it would be hard or time-consuming for you to do.

You still need to verify it, but "find the right things to read in the first place" is often a time intensive process in itself.

(You might, at that point, argue that "what if LLM fails to find a key article/paper/whatever", which I think is both a reasonable worry, and an unreasonable standard to apply. "What if your google search doesn't return it" is an obvious counterpoint, and I don't think you can make a reasonable argument that you journalists should be forced to cross-compare SERPs from Google/Bing/DuckDuckGo/AltaVista or whatever.)

Re: Our newsroom AI policy

#32

Post by intended »

legitster wrote: post #4

AI is in danger of peeing in it's own water source. It's unbelievably useful at imitating and generating content, but it needs enough original content to be able to train and scrape. Google got one thing wrong and nearly destroyed the internet - people need to have an incentive to contribute content online, and that incentive should not be to game the system for advertising. This in particular dawned on me when askin…

> in danger

It has already done so, and we can be confident in saying that.

Verified content will always be relatively expensive when compared to AI content.

Visits to wikipedia and most sites have dropped. Rtings has gone full paywall. Ad revenue for producing Verified content will be too meager to allow for public consumption.

Theres jokes about GenAI being the great filter; while I doubt this, I do hope this is the final push that makes us think of how we want our information commons to be nurtured.

Re: Our newsroom AI policy

#33

Post by intended »

vintagedave wrote: post #6

> Anyone who uses AI tools in our editorial workflow is responsible for the accuracy and integrity of the resulting work. This responsibility cannot be transferred to colleagues, editors... This sounds a direct callout to the incident earlier this year where an apparently sick staff member relied on an AI to reproduce quotes, and it did not. Ars retracted the article and the staffmember was fired. I have felt very et…

>This sounds a direct abrogation of journalistic standards by the Ars editorial team.

We depended on an ecosystem of news and journalism to keep our polities informed.

However, if that ecosystem is starving it will increasingly fail to live up to its standards and we can expect these failures to impact us increasingly.

I am not defending bad journalists, nor creating an excuse to tolerate such behavior in the future.

I am describing the macro trend we are facing, the failure state we can expect, and asking what happens if nothing grows to replace it.

The NYT earns revenue through games more than journalism and ads. Wikipedia is seeing reduced visitors due to AI summaries, and this leads to lower donations. A review site I used went into a full paywall.

I don't really see how Ars or most other sites will be able to earn revenue and pay salaries in this bot first environment.

Re: Our newsroom AI policy

#34

Post by enraged_camel »

applfanboysbgon wrote: post #8

Self-contradictory policy. > Reporters may use AI tools vetted and approved for our workflow to assist with research, including navigating large volumes of material, summarizing background documents, and searching datasets. If this is their official policy, Ars Technica bears as much responsibility as the author they fired for the fabricated reporting. LLMs are terrible at accurately summarizing anything. They very r…

[dead]

Re: Our newsroom AI policy

#35

Post by Mordisquitos »

JumpCrisscross wrote: post #19
applfanboysbgon wrote: post #13

Earlier quoted context omitted.

Any verification process thorough enough to catch all LLM fabrications would take more work than simply not using the LLM in the first place. If anything verifying what an LLM wrote is substantially more difficult than just reading the material it's "summarising", because you need to fully read and comprehend the material and then also keep in mind what the LLM generated to contrast and at that point what the fuck ar…

> Any verification process thorough enough to catch all LLM fabrications would take more work than simply not using the LLM in the first place Sometimes you have a weak hunch that may take hours to validate. Putting an LLM to doing the preliminary investigation on that can be fruitful. Particularly if, as if often the case, you don't have a weak hunch, but a small basket of them.

You can prompt LLMs to scan thousands of documents to generate text validating your hunches. In some cases those validated hunches may even be correct.

Re: Our newsroom AI policy

#36

Post by Mordisquitos »

fooker wrote: post #20
applfanboysbgon wrote: post #8

Self-contradictory policy. > Reporters may use AI tools vetted and approved for our workflow to assist with research, including navigating large volumes of material, summarizing background documents, and searching datasets. If this is their official policy, Ars Technica bears as much responsibility as the author they fired for the fabricated reporting. LLMs are terrible at accurately summarizing anything. They very r…

> LLMs are terrible at accurately summarizing anything. I think you are perhaps stuck in 2023?

And yet we are discussing this in the context of a reporter having been fired from Ars Technica for publishing an article which included inaccurate LLM-generated summaries in 2026. How come?

https://news.ycombinator.com/item?id=47226608

Re: Our newsroom AI policy

#37

Post by meander_water »

legitster wrote: post #4

AI is in danger of peeing in it's own water source. It's unbelievably useful at imitating and generating content, but it needs enough original content to be able to train and scrape. Google got one thing wrong and nearly destroyed the internet - people need to have an incentive to contribute content online, and that incentive should not be to game the system for advertising. This in particular dawned on me when askin…

I think most labs actively create synthetic data using existing model as part of the mix for the pretraining stage for their next model.

Would love to know exactly what the latest process is to keep slop out of training data.

Re: Our newsroom AI policy

#38

Post by bandrami »

Paracompact wrote: post #16
applfanboysbgon wrote: post #13

Earlier quoted context omitted.

Any verification process thorough enough to catch all LLM fabrications would take more work than simply not using the LLM in the first place. If anything verifying what an LLM wrote is substantially more difficult than just reading the material it's "summarising", because you need to fully read and comprehend the material and then also keep in mind what the LLM generated to contrast and at that point what the fuck ar…

> I believe this policy can never result in a positive outcome. I get where you're coming from (I'm learning more and more over time that every sentence or line of code I "trust" an AI with, will eventually come back to bite me), but this is too absolutist. Really, no positive result, ever, in any context? We need more nuanced understanding of this technology than "always good" or "always bad."

If you need accuracy, an LLM is not the tool for that use case. LLMs are for when you need plausibility. There are real use cases for that, but journalism is not one of them.

Re: Our newsroom AI policy

#39

Post by fooker »

Mordisquitos wrote: post #36
fooker wrote: post #20

Earlier quoted context omitted.

> LLMs are terrible at accurately summarizing anything. I think you are perhaps stuck in 2023?

And yet we are discussing this in the context of a reporter having been fired from Ars Technica for publishing an article which included inaccurate LLM-generated summaries in 2026. How come? https://news.ycombinator.com/item?id=47226608

Maybe you should read the article? :)

What failed was extracting verbatim quotes, not summarizing.

If you want an LLM to do verbatim anything, it has to be a tool call. So I’m not surprised.

Re: Our newsroom AI policy

#40

Post by furyofantares »

applfanboysbgon wrote: post #8

Self-contradictory policy. > Reporters may use AI tools vetted and approved for our workflow to assist with research, including navigating large volumes of material, summarizing background documents, and searching datasets. If this is their official policy, Ars Technica bears as much responsibility as the author they fired for the fabricated reporting. LLMs are terrible at accurately summarizing anything. They very r…

I'm not a journalist and just for random things I'm interested in, I have no problem using an LLM to point me in a direction and then directly engage with the source rather than treat any of the LLM output as authoritative. It's easy to do. This is not a flamethrower.
Post reply on HN