Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Improving recommendation systems and search in the age of LLMs

eugeneyan.com

Post reply on HN
Print view View original

71–80 of 98 posts

Re: Improving recommendation systems and search in the age of LLMs

#71

Post by jamesblonde »

drodgers wrote: post #38
jamesblonde wrote: post #12

It is very interesting that Eugene does this work and publishes it so soon after conferences. Traditionally this would be a literature survey by a PhD student and would take 12 months to come out as some obscure journal behind a walled garden. I wonder if it is an outlier (Eugene is good!) or a sign of things to come?

> a sign of things to come Isn't this, like, a sign of what's been happening for the last 20+ years (arxiv, blogs etc.)?

To some extent. But it's hard to find quality. Eugene's stuff is quality. For example, i'm in distributed systems, databases, and MLOps. Murat Demirbas (Uni Buffalo) has been the best in dist systems. Andy Pavlo (CMU) for databases. Stanford (Matei) have been doing the best summarizing in MLOps.

Re: Improving recommendation systems and search in the age of LLMs

#72

Post by syndacks »

softwaredoug wrote: post #70

A lot of teams can do a lot with search with just LLMs in the loop on query and index side doing enrichment that used to be months-long projects. Even with smaller, self hosted models and fairly naive prompts you can turn a search string into a more structured query - and cache the hell out of it. Or classify documents into a taxonomy. All backed by boring old lexical or vector search engine. In fact I’d say if you’r…

Can you share more, or at least point me in the right direction?

Re: Improving recommendation systems and search in the age of LLMs

#73

Post by braiamp »

x1xx wrote: post #34

> Spotify saw a 9% increase in exploratory intent queries, a 30% rise in maximum query length per user, and a 10% increase in average query length—this suggests the query recommendation updates helped users express more complex intents To me it's not clear that it should be interpreted as an improvement: what I read in this summary is that users had to search more and to enter longer queries to get to what they neede…

Yeah, this should be evaluated in a multivariate/bivariate model. Of the successful queries, how the length changed before and after interventions.

Re: Improving recommendation systems and search in the age of LLMs

#74

Post by MostlyStable »

wildrhythms wrote: post #69
x1xx wrote: post #34

> Spotify saw a 9% increase in exploratory intent queries, a 30% rise in maximum query length per user, and a 10% increase in average query length—this suggests the query recommendation updates helped users express more complex intents To me it's not clear that it should be interpreted as an improvement: what I read in this summary is that users had to search more and to enter longer queries to get to what they neede…

No you don't understand, more queries = more engagement!

It's relatively easy to construct a scenario where more search is in fact indicative of better search. To stick with Spotify: let's imagine they have an amazing search tool that consistently finds new, interesting music that the user genuinely likes. I can imagine that in that situation, users are going to search more, because doing so consistently gets them new, enjoyable music.

But the opposite is equally possible: a terrible search tool could regularly fail to find what the user is looking for or produce music that they enjoy. In this situation, I can also imagine users searching more, because it takes more search effort to find something they like.

They key is why are users searching. In Spotify's case I imagine that you could try and connect number of searches per listen, or how often a search results in a listen and how often those listens result in a positive rating. There are probably more options, but there needs to be some way of connecting the amount of search with how the user feels about those search results.

And yeah, using nothing other than search volume is probably a bad way to go about it

Re: Improving recommendation systems and search in the age of LLMs

#75

Post by osmarks »

amelius wrote: post #47
osmarks wrote: post #22

Earlier quoted context omitted.

You could just run a local LLM over every document and ask it "is this related to this query". I don't think you actually want to wait a week (and holding all the documents you might ever want to search would run to petabytes). (the reasonable way is embedding search, which runs much faster with some precomputation, but you still have to store things)

A better way would be to ask the LLM to generate keywords (or queries). And then use old school techniques to find a set of documents, and then filter those using another LLM.

https://arxiv.org/abs/2212.10496

Re: Improving recommendation systems and search in the age of LLMs

#78

Post by bradly »

pizza wrote: post #19
anonymousDan wrote: post #9

It's interesting that none of these papers seem to be coming out of academic labs....

Checking if a recommendation system is actually good in practice is kind of tough to do without owning a whole internet media platform as well. At best, you'll get the table scraps from these corporations (in the form of toy datasets/models made available), and you still will struggle to make your dev loop productive enough without throwing similar amounts of compute that the ~FAANGs do so as to validate whether that…

> you still will struggle to make your dev loop productive enough without throwing similar amounts of compute that the ~FAANGs do so as to validate whether that 0.2% improvement you got really meant anything or not

And do not forget the incredible of number of actual humans FAANG pays every day to evaluate any changes in result sets for top x,000 queries.

Re: Improving recommendation systems and search in the age of LLMs

#79

Post by pizza »

dmbche wrote: post #27
nine_k wrote: post #5

Earlier quoted context omitted.

> you can just search your files using your prefered file explorer This only work if you remember specific substrings. An LLM (or some other language model) can summarize and interpolate. It can be asked to find that file that mentions a transaction for buying candy, and it has a fair chance to find it, even if none of the words "transaction", "buying" or "candy" are present in the file, e.g. it says "shelled out $17…

But isn't that candy example non-sensical? In what situation do you need some information without any of the context(or without knowing any of the context)? i really believe that this is not an actual problem in need of solving, but instead creating a tool (personal ai assistant) and trying to find a usecase Edit0: note to self, rambling - assuming there exist valuable information that one needa to access in their fi…

Here’s an example of a type of feature I want: I’m looking at a menu from a popular restaurant and it has hundreds of choices. I start to feel some analysis paralysis. I say to my computer, “hey computer, I’m open to any suggestions, so long as it’s well-seasoned, spicy, salty, has some protein and fiber, easy to digest, rich in nutrients, not too dry, not too oily, pairs well with , etc..” Basically, property-oriented search queries whose answers can be verified, without having to trudge through them myself, where I don’t really care about correctness, just satisficing.

Re: Improving recommendation systems and search in the age of LLMs

#80

Post by ntonozzi »

syndacks wrote: post #72
softwaredoug wrote: post #70

A lot of teams can do a lot with search with just LLMs in the loop on query and index side doing enrichment that used to be months-long projects. Even with smaller, self hosted models and fairly naive prompts you can turn a search string into a more structured query - and cache the hell out of it. Or classify documents into a taxonomy. All backed by boring old lexical or vector search engine. In fact I’d say if you’r…

Can you share more, or at least point me in the right direction?

One place to explore more would be Doc2Query: https://arxiv.org/abs/1904.08375.

It’s not the latest and hottest but super simple to do with LLMs these days and can improve a lexical search engine quite a lot.

Post reply on HN