Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Improving recommendation systems and search in the age of LLMs

eugeneyan.com

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Improving recommendation systems and search in the age of LLMs

#31

Post by osmarks »

kortilla wrote: post #29
osmarks wrote: post #22

Earlier quoted context omitted.

You could just run a local LLM over every document and ask it "is this related to this query". I don't think you actually want to wait a week (and holding all the documents you might ever want to search would run to petabytes). (the reasonable way is embedding search, which runs much faster with some precomputation, but you still have to store things)

The entire library of Congress is like 10TB. You don’t need anything near petabytes until you get out of text into rich media.

Common Crawl is petabytes. Anna's Archive is about a petabyte, but it includes PDFs with images.

Re: Improving recommendation systems and search in the age of LLMs

#32

Post by acchow »

dmbche wrote: post #3
whatever1 wrote: post #2

Why we don’t have an LLM based search tool for our pc / smartphones? Specially for the smartphones all of your data is on the cloud anyway, instead of just scraping it for advertising and the FBI they could also do something useful for the user?

It doesn't solve any problem, you can just search your files using your prefered file explorer (crtl-f) I'd assume most people organise their files so that they know where things are as well.

Curiously, the things I search most often are not located in files: calendar, photo content/location, email, ChatGPT history, Spotify library, iMessage/whatsapp history, contacts, notes, Amazon order history

Re: Improving recommendation systems and search in the age of LLMs

#34

Post by x1xx »

> Spotify saw a 9% increase in exploratory intent queries, a 30% rise in maximum query length per user, and a 10% increase in average query length—this suggests the query recommendation updates helped users express more complex intents

To me it's not clear that it should be interpreted as an improvement: what I read in this summary is that users had to search more and to enter longer queries to get to what they needed.

Re: Improving recommendation systems and search in the age of LLMs

#35

Post by rorytbyrne »

x1xx wrote: post #34

> Spotify saw a 9% increase in exploratory intent queries, a 30% rise in maximum query length per user, and a 10% increase in average query length—this suggests the query recommendation updates helped users express more complex intents To me it's not clear that it should be interpreted as an improvement: what I read in this summary is that users had to search more and to enter longer queries to get to what they neede…

We would need to normalise query length by the success rate to draw any informative conclusions here. The rate of immediate follow-up queries could be a decent proxy for this.

Re: Improving recommendation systems and search in the age of LLMs

#36

Post by GraemeMeyer »

whatever1 wrote: post #2

Why we don’t have an LLM based search tool for our pc / smartphones? Specially for the smartphones all of your data is on the cloud anyway, instead of just scraping it for advertising and the FBI they could also do something useful for the user?

It's coming for PCs soon: https://www.theregister.com/2025/01/20/microsoft_unveils_win...

And to a certain extent for the Microsoft cloud experience as well: https://www.theverge.com/2024/10/8/24265312/microsoft-onedri...

Re: Improving recommendation systems and search in the age of LLMs

#37

Post by Traubenfuchs »

x1xx wrote: post #34

> Spotify saw a 9% increase in exploratory intent queries, a 30% rise in maximum query length per user, and a 10% increase in average query length—this suggests the query recommendation updates helped users express more complex intents To me it's not clear that it should be interpreted as an improvement: what I read in this summary is that users had to search more and to enter longer queries to get to what they neede…

> a 9% increase in exploratory intent queries

Users struggle to find the right stuff or stuff that‘s so good they don‘t need do do more queries.

> a 30% rise in maximum query length per user, and a 10% increase in average query length

Users need to execute more complex queries to find what they are looking for.

Re: Improving recommendation systems and search in the age of LLMs

#38

Post by drodgers »

jamesblonde wrote: post #12

It is very interesting that Eugene does this work and publishes it so soon after conferences. Traditionally this would be a literature survey by a PhD student and would take 12 months to come out as some obscure journal behind a walled garden. I wonder if it is an outlier (Eugene is good!) or a sign of things to come?

> a sign of things to come

Isn't this, like, a sign of what's been happening for the last 20+ years (arxiv, blogs etc.)?

Re: Improving recommendation systems and search in the age of LLMs

#39

Post by desdenova »

thorum wrote: post #16

In the age of local LLMs I’d like to see a personal recommendation system that doesn’t care about being scalable and efficient. Why can’t I write a prompt that describes exactly what I’m looking for in detail and then let my GPU run for a week until it finds something that matches?

Why can't you?

Just run the biggest model you can find out of swap and wait a long time for it to finish.

You'll obviously see more focus on smaller models, because most people aren't willing to wait weeks for their slop, and also don't have server GPU clusters to run huge models.

Post reply on HN