Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Working with The Associated Press to provide fresh results for the Gemini app

blog.google

Post reply on HN
Print view View original

21–30 of 68 posts

Re: Working with The Associated Press to provide fresh results for the Gemini app

#21

Post by CuriouslyC »

notatoad wrote: post #19
dismalaf wrote: post #8

Earlier quoted context omitted.

> I can only hope this data is being incorporated in some way that makes hallucinations less likely. The key word is "real-time". LLMs can't be trained in realtime, so it's obviously going to call an API that pulls up and reads from AP news, just like their search engine.

i don't think you can assume that - "real time" in this context could just mean they feed every article into their training system as soon as it's published.

They can't deploy that fast and people want to pin model version so it's not feasible anyhow.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#22

Post by iamjackg »

notatoad wrote: post #19
dismalaf wrote: post #8

Earlier quoted context omitted.

> I can only hope this data is being incorporated in some way that makes hallucinations less likely. The key word is "real-time". LLMs can't be trained in realtime, so it's obviously going to call an API that pulls up and reads from AP news, just like their search engine.

i don't think you can assume that - "real time" in this context could just mean they feed every article into their training system as soon as it's published.

That seems more unlikely to me -- training is not free and takes a long time, so it would not result in "[enhancing] the usefulness of results displayed in the Gemini app" and it being "particularly helpful to our users looking for up-to-date information."

Fine-tuning, which is cheaper and faster, has been proven to not be a good solution to "teach" models new facts.

I think what's most likely here is that Gemini will have access to a form of RAG based on a database of AP articles that gets updated in real-time as new articles are published.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#23

Post by jonas21 »

stusmall wrote: post #9
onlyrealcuzzo wrote: post #4

Earlier quoted context omitted.

Gemini is the leading model with the lowest hallucination rate: https://www.visualcapitalist.com/ranked-ai-models-with-the-l... I would expect that number to go down from 1.3% to below 1% over the course of the year. There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident. Realistically, hardly anything is 100% accurate besides math.

1.3% isn't great. I'd rather just go, and pay, directly to trusted news sources. Everyone has different tolerance for falsehoods and priorities I guess.

What's the error rate for human journalists? Based on my experience, I'd guess it's much higher than 1.3%.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#24

Post by kccqzy »

simonw wrote: post #6
onlyrealcuzzo wrote: post #4

Earlier quoted context omitted.

Gemini is the leading model with the lowest hallucination rate: https://www.visualcapitalist.com/ranked-ai-models-with-the-l... I would expect that number to go down from 1.3% to below 1% over the course of the year. There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident. Realistically, hardly anything is 100% accurate besides math.

The Gemini models themselves may score well on this, but Google's feature implementations are a whole other thing. AI Overviews frequently take untrustworthy search results (like a fan fiction plot outline for Encanto 2) and turn those into confidently incorrect answers. https://simonwillison.net/2024/Dec/29/encanto-2/

And doesn't bringing in The Associated Press solve this problem? No need for the AI to decide what is trustworthy or not. For the vast majority of people everything The Associated Press publishes is trustworthy.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#25

Post by dismalaf »

notatoad wrote: post #19
dismalaf wrote: post #8

Earlier quoted context omitted.

> I can only hope this data is being incorporated in some way that makes hallucinations less likely. The key word is "real-time". LLMs can't be trained in realtime, so it's obviously going to call an API that pulls up and reads from AP news, just like their search engine.

i don't think you can assume that - "real time" in this context could just mean they feed every article into their training system as soon as it's published.

Have you ever asked an LLM what time it is? It takes months to train them...

Re: Working with The Associated Press to provide fresh results for the Gemini app

#26

Post by marsavar »

stusmall wrote: post #9
onlyrealcuzzo wrote: post #4

Earlier quoted context omitted.

Gemini is the leading model with the lowest hallucination rate: https://www.visualcapitalist.com/ranked-ai-models-with-the-l... I would expect that number to go down from 1.3% to below 1% over the course of the year. There's always a chance what you're reading is wrong - due to purposeful deception, negligence, or accident. Realistically, hardly anything is 100% accurate besides math.

1.3% isn't great. I'd rather just go, and pay, directly to trusted news sources. Everyone has different tolerance for falsehoods and priorities I guess.

Anything other than 0% is borderline immoral. Imagine sending a push notification to somebody's phone with a completely made-up headline summary. Even if it happens once in a hundred times, that's too much. Things like that slowly but surely erode trust and make it harder and harder to trust anything that's generated by AI, especially when it comes to news, where trustworthiness is essential, and probably the main reason people pay for news. See for example https://www.bbc.co.uk/news/articles/cge93de21n0o

Re: Working with The Associated Press to provide fresh results for the Gemini app

#27

Post by tomrod »

dismalaf wrote: post #25
notatoad wrote: post #19

Earlier quoted context omitted.

i don't think you can assume that - "real time" in this context could just mean they feed every article into their training system as soon as it's published.

Have you ever asked an LLM what time it is? It takes months to train them...

But it can be trained to access basic, limited APIs to get current information.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#28

Post by summerlight »

notatoad wrote: post #19
dismalaf wrote: post #8

Earlier quoted context omitted.

> I can only hope this data is being incorporated in some way that makes hallucinations less likely. The key word is "real-time". LLMs can't be trained in realtime, so it's obviously going to call an API that pulls up and reads from AP news, just like their search engine.

i don't think you can assume that - "real time" in this context could just mean they feed every article into their training system as soon as it's published.

If there's any company who can afford "real-time LLM training" at this moment, I'm 100% sure they will win this AI race since they probably have at least ~10x compute compared to competitors. Of course, no one can do that right now.

Re: Working with The Associated Press to provide fresh results for the Gemini app

#29

Post by scarface_74 »

itsibitzi wrote: post #2

As someone who works in the news industry I find it pretty sad that we've just capitulated to big tech on this one. There are countless examples of AI summaries getting things catastrophically wrong, but I guess Google has long since decided that pushing AI was more important than accurate or relevant results, as can also be seen with their search results that simply omit parts of your query. I can only hope this dat…

The examples that have made news were with iOS. iOS doesn’t really do a summary of the content. It just tries to do a summary of the headline.

The on device model that it uses is also literally 1% the size of the large models like Gemini

Post reply on HN