Live data from Hacker News

Working with The Associated Press to provide fresh results for the Gemini app

blog.google

31–40 of 68 posts

Re: Working with The Associated Press to provide fresh results for the Gemini app

#31
post #23
post #9

Earlier quoted context omitted.

1.3% isn't great. I'd rather just go, and pay, directly to trusted news sources. Everyone has different tolerance for falsehoods and priorities I guess.

What's the error rate for human journalists? Based on my experience, I'd guess it's much higher than 1.3%.

Except when that happens, a clarification is almost always added at the bottom of the article ("This article was amended on [date]. An earlier version said xxx" or some variation thereof). You're not gonna get a second push notification from an AI summary saying "Oopsies, the previous notification was wrong". Once it's out, it's out, and that sort of damage is difficult to repair.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#32

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

Gemini is the leading model with the lowest hallucination rate: https://www.visualcapitalist.com/ranked-ai-models-with-the-l... I would expect that number to go down from 1.3% to below 1% over the course of the year. There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident. Realistically, hardly anything is 100% accurate besides math.

> There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident.

I am quite certain my personal hallucinations level is more than 1.3%, obviously we want our machines to be better than us, but my doctor once said folic acid is not a vitamin.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#33
post #7

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

Journalism is dying, killed by its own excesses and by the internet. Google is offering it life support. The other option is death.

"killed by its own excesses"?

Re: Working with The Associated Press to provide fresh results for the Gemini app

#34
wow! this sure is great! gemini has worked so great up until this point - for example, i learned that a man who died in 1850 is one of three private owners of the airbus a340-600 last week! i'm so glad gemini exists and i absolutely cannot wait to experience a world wherein people get news from it.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#35

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

Unfortunately this has just been the reality over the last couple years. People just ignore the hallucination problem (or try to say it isn't a big deal). And yet we have seen time and time again examples of these models being given something, told to summarize it, and still hallucinate important details. So you can't even make the argument that its data is flawed or something.

These models will interject information from their training whether or it is relevant or not. This is just due to the nature of how these models work.

Anyone trying to argue that it doesn't happen that often or anything is missing the key problem. Sure it may be right most of the time, but all that does is build a false sense of security and eventually you stop double checking or clicking through to a source. Whether it is a search result, manipulating data, or whatever.

This is made infinitely worse when these summaries are one and done, a single user is going to see the output and no one else will see it to fact check. It isn't like an article being wrong that everyone reading it is reading the same article, can then comment that something is wrong, it get updated, and so on and so forth. That feedback loop is non-existent with these models

Re: Working with The Associated Press to provide fresh results for the Gemini app

#36

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

Gemini is the leading model with the lowest hallucination rate: https://www.visualcapitalist.com/ranked-ai-models-with-the-l... I would expect that number to go down from 1.3% to below 1% over the course of the year. There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident. Realistically, hardly anything is 100% accurate besides math.

I think people really don't understand the effort, care and risk that goes into producing quality reporting.

I work with investigative reporters on stories that take many months to produce. Every time we receive a leak there is an extensive process of proving public interest before we can even start looking at the material. Once we can see it in we have to be extremely careful with everything we note down to make sure that our work isn't seen as prejudiced if legal discovery happens. We're constantly going back and forth with our editorial legal team to make sure what we're saying is fair and accurate. And in the end, the people we're reporting are given a chance to refute any of the facts we're about to present. Any mistakes can result in legal action that can ruin the lives of reporters and shut down companies.

Now, imagine I were to go to a reporter who has spent 6 months working on a story about, for example, a high profile celebrity sexually assaulted multiple women, how the royal family hides their wealth and are exempt from laws, or how multinational corporations use legal loopholes to avoid paying taxes, and said, "oh, 1% of people reading this will likely be given some totally made up details".

Given that stories often have more than a million impressions, this would lead tens of thousands of people with potentially libellous "hallucinations".

It simply should not be allowed.

LLMs have their place, for sure, but presenting the news is not it.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#37

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

The news industry capitulated to big tech the moment it got reliant on big tech for the majority of its revenue. The entire media landscape today is the direct result of that.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#38

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

It's responsibility laundering — AI can say whatever they want and they can shrug it off by saying bots are sometimes unreliable

Re: Working with The Associated Press to provide fresh results for the Gemini app

#39
post #9

Earlier quoted context omitted.

1.3% isn't great. I'd rather just go, and pay, directly to trusted news sources. Everyone has different tolerance for falsehoods and priorities I guess.

Anything other than 0% is borderline immoral. Imagine sending a push notification to somebody's phone with a completely made-up headline summary. Even if it happens once in a hundred times, that's too much. Things like that slowly but surely erode trust and make it harder and harder to trust anything that's generated by AI, especially when it comes to news, where trustworthiness is essential, and probably the main re…

Have you ever read a news article on a subject where you have expertise and knew it was inaccurate? The news is probably more inaccurate than you think.

I bet you think the news is accurate all other times. It’s called “Gell-Mann Amnesia”

Re: Working with The Associated Press to provide fresh results for the Gemini app

#40

Earlier quoted context omitted.

Gemini is the leading model with the lowest hallucination rate: https://www.visualcapitalist.com/ranked-ai-models-with-the-l... I would expect that number to go down from 1.3% to below 1% over the course of the year. There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident. Realistically, hardly anything is 100% accurate besides math.

I think people really don't understand the effort, care and risk that goes into producing quality reporting. I work with investigative reporters on stories that take many months to produce. Every time we receive a leak there is an extensive process of proving public interest before we can even start looking at the material. Once we can see it in we have to be extremely careful with everything we note down to make sur…

Although I agree with every single sentence you've said, we've seen in the past decade how only very small percentage of people actually care about the content of the news. Everyone just discusses and gets their information from the headlines, so this is a natural consequence of "let's just summarize it to a couple of sentences since nobody reads it anyways".
Post reply on HN