Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ars Technica fires reporter after AI controversy involving fabricated quotes

futurism.com

Post reply on HN
Print view View original

71–80 of 409 posts

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#71

Post by esperent »

petterroea wrote: post #62
AnonC wrote: post #50

Journalists and bloggers usually write about others’ mess ups and apologies, dissecting which apologies are authentic and which apologies are non-apologies. In this incident, Aurich Lawson of Ars Technica deleted the original article (which had LLM hallucinated quotes) instead of updating it with the error. He then published a vague non-apology, just like large companies and politicians usually do. And now we learn t…

It seemed to me like very hasty self defense, there's a lot of AI slop hate and Ars can't risk becoming known as slop when their readers are probably prone to be aware of the issue. I don't think Ars thought they had a choice but to cut off the journalist who made the mistake, especially when it was regarding a very touchy subject. I don't think they had a choice, it's impossible for us readers to know if this was a…

All they had to do was write a clear and simple message saying that one of their staff was responsible, has been fired, and they'll take steps to avoid this in future.

Their actions so far just make me think they're panicking and found a scapegoat to blame it on, but they're not going to put any new checks in place so it'll just happen again.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#72

Post by deathanatos »

raincole wrote: post #39

I have to admit, nowadays Google AI Overview's accuracy is so good that I often don't check the links. It's scary that it got from 'practically useless' to 'the actual google search' in less than two years. I really don't know where the internet is heading to and how any content site can survive.

You should be checking the links more often, IMO. I've seen it respond a number of times with content that is not supported by the citations.

While trying to find an example by going back through my history though, the search "linux shebang argument splitting" comes back from the AI with:

> On Linux and most Unix-like systems, the shebang line (e.g., #!/bin/bash ...) does not perform argument splitting by default. The entire string after the interpreter path is passed as a single argument to the interpreter.

(that's correct) …followed by:

> To pass multiple arguments portably on modern systems, the env command with the -S (split string) option is the standard solution.

(`env -S` isn't portable. IDK if a subset of it is portable, or not. I tend to avoid it, as it is just too complex, but let's call "is portable" opinion.)

(edited out a bit about the splitting on Linux; I think I had a different output earlier saying it would split the args into "-S" and "the rest", but this one was fine.)

> Note: The -S option is a modern extension and may not be available

But this, … which is it.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#73

Post by weird-eye-issue »

jmyeet wrote: post #28

The crazy part to me is that even here on HN there are people who still insist that LLMs don't fabricate things or otherwise lie. I wonder if these are the same people who 3-4 years ago were insisting putting 20 characters onto a blockchain (ie an NFT, which was just a URL) was the next multi-billion dollar business. Sure there is such a thing as a naysayer but there are also people think all forms of valid criticism…

I've never seen anyone here claim that AI never hallucinates or can't provide incorrect information.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#75

Post by gertop »

petterroea wrote: post #62
AnonC wrote: post #50

Journalists and bloggers usually write about others’ mess ups and apologies, dissecting which apologies are authentic and which apologies are non-apologies. In this incident, Aurich Lawson of Ars Technica deleted the original article (which had LLM hallucinated quotes) instead of updating it with the error. He then published a vague non-apology, just like large companies and politicians usually do. And now we learn t…

It seemed to me like very hasty self defense, there's a lot of AI slop hate and Ars can't risk becoming known as slop when their readers are probably prone to be aware of the issue. I don't think Ars thought they had a choice but to cut off the journalist who made the mistake, especially when it was regarding a very touchy subject. I don't think they had a choice, it's impossible for us readers to know if this was a…

AnonC doesn't seem to be upset that the journalist was fired. The disappointment comes from Ars trying to brush this entire situation away by deleting articles, comments, and making no statement on their website.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#76

Post by archagon »

raincole wrote: post #39

I have to admit, nowadays Google AI Overview's accuracy is so good that I often don't check the links. It's scary that it got from 'practically useless' to 'the actual google search' in less than two years. I really don't know where the internet is heading to and how any content site can survive.

Uh, really? In my experience, at least a quarter of the info it gives me is usually manufactured or incorrect in some critical way.

In fact, if you switch to "Pro" mode, it frequently says the complete opposite of what it claimed in "Fast" mode while still being ~10-20% wrong. (Not to say it's not useful — there's no better way to aggregate and synthesize obscure information — but it should definitely not be relied on a source of anything other than links for detailed followup.)

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#77

Post by lovich »

aizk wrote: post #26

I have a story with Benji. Last year I went viral, and Benji was the first person to interview me. It was a really cool experience, we chatted via Twitter dms, and he wrote a piece about my work - overall did a decent job. Then, 6 months later a separate project I was adjacent to was starting to pick up steam. I reached out to him asking if he wanted to cover us. No response. Then, tech crunch wrote an article on our…

[flagged]

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#78

Post by petterroea »

gertop wrote: post #75
petterroea wrote: post #62

Earlier quoted context omitted.

It seemed to me like very hasty self defense, there's a lot of AI slop hate and Ars can't risk becoming known as slop when their readers are probably prone to be aware of the issue. I don't think Ars thought they had a choice but to cut off the journalist who made the mistake, especially when it was regarding a very touchy subject. I don't think they had a choice, it's impossible for us readers to know if this was a…

AnonC doesn't seem to be upset that the journalist was fired. The disappointment comes from Ars trying to brush this entire situation away by deleting articles, comments, and making no statement on their website.

My understanding is that AnonC is upset at Ars not taking the mature approach by allowing this to become a learning moment for the employee and using it to double down and confirm their stance on AI generated content. There's strength in maturity. But I am doing some reading between the lines, and I'm possibly reading a bit too much into "There’s something to be said about the value of owning up to issues"

Reminds me of a story I was told as an intern deploying infra changes to prod for the first time. Some guy had accidentally caused hours of downtime, and was expecting to be fired, only for his boss to say "Those hours of downtime is the price we pay to train our staff (you) to be careful. If we fire you, we throw the investment out the window"

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#79

Post by gertop »

jmyeet wrote: post #28

The crazy part to me is that even here on HN there are people who still insist that LLMs don't fabricate things or otherwise lie. I wonder if these are the same people who 3-4 years ago were insisting putting 20 characters onto a blockchain (ie an NFT, which was just a URL) was the next multi-billion dollar business. Sure there is such a thing as a naysayer but there are also people think all forms of valid criticism…

I've not heard many people claim that LLMs don't hallucinate, however I have seen people (that I previously believed to be smart):

1. Believe LLMs outright even knowing they are frequently wrong

2. Claim that LLMs making shit up is caused by the user not prompting it correctly. I suppose in the same way that C is memory safe and only bad programmers make it not so.

Re: Ars Technica fires reporter after AI controversy involving fabricated quotes

#80

Post by raincole »

SchemaLoad wrote: post #49
raincole wrote: post #39

I have to admit, nowadays Google AI Overview's accuracy is so good that I often don't check the links. It's scary that it got from 'practically useless' to 'the actual google search' in less than two years. I really don't know where the internet is heading to and how any content site can survive.

It's because the AI overview is most of the time directly summarising the search results rather than synthesizing an answer from internal model knowledge. Which is why it can hyperlink the sources for the facts now. Even a very dumb lightweight model can extract relevant text from articles I just can't see how this is sustainable since they are stealing from the sources who are now getting defunded.

> I just can't see how this is sustainable since they are stealing from the sources who are now getting defunded.

Yeah, that's why I said I don't know where the internet is heading to.

Post reply on HN