tl;dr: We apologize for getting caught. Ars Subscriptors in the comments thank Ars for their diligence in handling an editorial fuckup that wasn't identified by Ars.
I don't know how you could possibly have that take away from reading this. They did a review of their context to confirm this was an isolated incident and reaffirmed that it did not follow the journalistic standards they have set for themselves. They admit wrong doing here and point to multiple policy violations.
> That rule is not optional, and it was not followed here.
It’s not optional, but wasn’t followed, with zero repercussions.
What are they changing to prevent this from happening in the future? Why was the use of LLMs not disclosed in the original article? Do they host any other articles covertly generated by LLMs?
As far as I can tell, the pulled article had no obvious tells and was caught only because the quotes were entirely made up. Surely it's not the only one, though?
I don’t see how you could know that without more information. Using an AI tool doesn’t imply that they thought it would make up quotes. It might just be careless. Assuming malice without investigating is itself careless.
we are fucking doomed holy shit we're really at the point where people are just writing off a journalist passing off their job to a chatgpt prompt as though that's a normal and defensible thing to be doing
No one said it was defensible. They drew a distinction between incompetence and malice. Let's not misquote each other here in the comments.
People put a lot of weight on blame-free post-mortems and not punishing people who make "mistakes", but I believe that has to stop at the level of malice. Falsifying quotes is malice. Fire the malicious party or everything else you say is worthless.
There’s no malice if there was no intention of falsifying quotes. Using a flawed tool doesn’t count as intention.
The issues with such tools are highly documented though. If you’re going to use a tool with known issues you’d better do your best to cover for them.
People put a lot of weight on blame-free post-mortems and not punishing people who make "mistakes", but I believe that has to stop at the level of malice. Falsifying quotes is malice. Fire the malicious party or everything else you say is worthless.
There’s no malice if there was no intention of falsifying quotes. Using a flawed tool doesn’t count as intention.
Outsourcing writing to a bot without attribution may not be malicious, but it does strain integrity.
There’s no malice if there was no intention of falsifying quotes. Using a flawed tool doesn’t count as intention.
Outsourcing your job as a journalist to a chatbot that you know for a fact falsifies quotes (and everything else it generates) is absolutely intentional.
It's intentionally reckless, not intentionally harmful or intentionally falsifying quotes. I am sure they would have preferred if it hadn't falsified any quotes.
we are fucking doomed holy shit we're really at the point where people are just writing off a journalist passing off their job to a chatgpt prompt as though that's a normal and defensible thing to be doing
No one said it was defensible. They drew a distinction between incompetence and malice. Let's not misquote each other here in the comments.
Even if it didn't fabricate quotes wholesale, taking an LLM's output and claiming it as your own writing is textbook plagiarism, which is malicious intent. Then, if you know that LLMs are next-token-prediction-engines that have no concept of "truth" and are programmed solely to generate probabilistically-likely text with no specific mechanism of anchoring to "reality" or "facts", and you use that output in a journal that (ostensibly) exists for the reason of presenting factual information to readers, you are engaging in a second layer of malicious intent. It would take an astounding level of incompetence for a tech journal writer to not be aware of the fact that LLMs do not generate factual output reliably, and it beggars belief given that one of the authors has worked at Ars for 14 years. If they are that incompetent, they should probably be fired on that basis anyways. But even if they are that incompetent, that still only covers one half of their malicious intent.