Live data from Hacker News

Our newsroom AI policy

arstechnica.com

61–70 of 144 posts

Re: Our newsroom AI policy

#61

Self-contradictory policy. > Reporters may use AI tools vetted and approved for our workflow to assist with research, including navigating large volumes of material, summarizing background documents, and searching datasets. If this is their official policy, Ars Technica bears as much responsibility as the author they fired for the fabricated reporting. LLMs are terrible at accurately summarizing anything. They very r…

> the author they fired for the fabricated reporting Didn't one of the magazine's editors share the byline?

Yes but editors don't take the fall, they take the credit.

Everything occurred exactly as predicted.

Re: Our newsroom AI policy

#62
> AI tools must not be used to generate, extract, or summarize material that is then attributed to a named source, whether as a direct quote, a paraphrase, or a characterization of someone’s views.

This sounds overfit to their earlier incident.

Besides, I expect they already have a policy on accurate quotes.

> Anyone who uses AI tools in our editorial workflow is responsible for the accuracy and integrity of the resulting work.

AFAICT that's the actual simplified policy. Reasonable!

Re: Our newsroom AI policy

#63
post #55

From TFA: > Our approach comes from two convictions: Uhuh. > that AI cannot replace human insight, creativity, and ingenuity Sure, agreed, no dispute. > and that these tools, used well, can help professionals do better work. [[citation needed]] Prove it. I don't believe you.

If AI slop-generation helps your work, then your work was at least partially generating slop.

Slop may be useful, helpful, any number of things, but if you're using slop you're using slop.

Re: Our newsroom AI policy

#64

> Anyone who uses AI tools in our editorial workflow is responsible for the accuracy and integrity of the resulting work. This responsibility cannot be transferred to colleagues, editors... This sounds a direct callout to the incident earlier this year where an apparently sick staff member relied on an AI to reproduce quotes, and it did not. Ars retracted the article and the staffmember was fired. I have felt very et…

>This sounds a direct abrogation of journalistic standards by the Ars editorial team. We depended on an ecosystem of news and journalism to keep our polities informed. However, if that ecosystem is starving it will increasingly fail to live up to its standards and we can expect these failures to impact us increasingly. I am not defending bad journalists, nor creating an excuse to tolerate such behavior in the future.…

>We depended on an ecosystem of news and journalism to keep our polities informed.

If this is true and necessary we might as well skip the middleman and have the news and journalists run the polities.

Re: Our newsroom AI policy

#65

Earlier quoted context omitted.

>This sounds a direct abrogation of journalistic standards by the Ars editorial team. We depended on an ecosystem of news and journalism to keep our polities informed. However, if that ecosystem is starving it will increasingly fail to live up to its standards and we can expect these failures to impact us increasingly. I am not defending bad journalists, nor creating an excuse to tolerate such behavior in the future.…

Ars has a decently pricey direct subscription, doesn't it? With a lot of tech focused features included. Their strategy is probably the best you could set up in this ecosystem.

If it isn't clear from this policy that Ars is run by the advertisers and not the subscribers, I don't know what would make it clear.

Advertisers only care about eyeballs and really bad press; AI increases the first and rarely causes the second.

Re: Our newsroom AI policy

#66

Earlier quoted context omitted.

I think most labs actively create synthetic data using existing model as part of the mix for the pretraining stage for their next model. Would love to know exactly what the latest process is to keep slop out of training data.

const isAiContent = (str) => str.includes('—');? :)

Latest generation LLM's use en dashes instead of em dashes to avoid detection.

Re: Our newsroom AI policy

#67

AI is in danger of peeing in it's own water source. It's unbelievably useful at imitating and generating content, but it needs enough original content to be able to train and scrape. Google got one thing wrong and nearly destroyed the internet - people need to have an incentive to contribute content online, and that incentive should not be to game the system for advertising. This in particular dawned on me when askin…

I think most labs actively create synthetic data using existing model as part of the mix for the pretraining stage for their next model. Would love to know exactly what the latest process is to keep slop out of training data.

I think everyone overblows the whole "AI is poisoning AI!" thing. It could be a problem but the genuine value in Reddit or any other human social media is honestly pretty low from my estimates. It's great for seeing how humans talk but in terms of 'nutritional' value for truth or answers... I am not sold. If I was choosing what to 'feed' AI, I wouldn't even bother with textual social media (besides Github / Gitlab / other source control)

There's way more value, if seeking out answers, in following the links to external sources, scraping books, and other sources that aren't "unwashed masses saying whatever they want".

Re: Our newsroom AI policy

#68
AI for writing feels like such a stupid idea.

When you write things you just need to think about what you want to say and write it. It's not hard.

If you make an LLM generate the text, and you still bear responsibility for whether it's correct or not, you haven't actually reduced the amount of work you have to do at all because now instead of "thinking -> writing" you need "reading -> thinking -> rewriting."

"Reading is easier than writing" sounds like something that someone who never writes anything would think. In both code and natural languages writing is easier than reading. Text is after all a representation of thought, so to write you just need to represent your thoughts in a way others can understand, but in order to read you must decipher what the author was trying to convey from their written word. If the author is an LLM that is going to be a lot more difficult and annoying.

I've had similar feelings about LLM's "completing" code. It "feels" like I'm more productive, but at the same time the "completions" feel more like constant interjections that won't let me get into a flow of writing, and often when things don't work it's because the AI quickly generated code that looked correct at first glance, so I left it that way, but when I tried to run the program it turns out it was wrong, and when it happens I can't help but think: of course it's wrong, I haven't written the code yet. There is code, the LLM wrote it. But I haven't written it. I haven't done the work. So it's like having a "placeholder" that you still HAVE to check but there is no metadata that says this is a WIP, no # TODO: check the LLM output. It's a placeholder camouflaged as finished work.

Re: Our newsroom AI policy

#69
post #31

Earlier quoted context omitted.

Any verification process thorough enough to catch all LLM fabrications would take more work than simply not using the LLM in the first place. If anything verifying what an LLM wrote is substantially more difficult than just reading the material it's "summarising", because you need to fully read and comprehend the material and then also keep in mind what the LLM generated to contrast and at that point what the fuck ar…

The LLM can find material that it would be hard or time-consuming for you to do. You still need to verify it, but "find the right things to read in the first place" is often a time intensive process in itself. (You might, at that point, argue that "what if LLM fails to find a key article/paper/whatever", which I think is both a reasonable worry, and an unreasonable standard to apply. "What if your google search doesn…

I believe what their point is is that if you give people a "extract-needle-from-haystack" machine and then tell them they have to manually find where in the haystack the needle was, it defeats the purpose of having the machine.

With that said, a good RAG solution would come with metadata to point to where it was sourced from.

Re: Our newsroom AI policy

#70

Earlier quoted context omitted.

> Any verification process thorough enough to catch all LLM fabrications would take more work than simply not using the LLM in the first place Sometimes you have a weak hunch that may take hours to validate. Putting an LLM to doing the preliminary investigation on that can be fruitful. Particularly if, as if often the case, you don't have a weak hunch, but a small basket of them.

You can prompt LLMs to scan thousands of documents to generate text validating your hunches. In some cases those validated hunches may even be correct.

It's easy to get an LLM to make any argument you like based on whatever data is available. Those arguments are going to be trivially bad if that data is bad.
Post reply on HN