Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.
A 2s LLM call is pretty slow.
Don't classify, hallucinate
11–20 of 118 posts
Re: Don't classify, hallucinate
#12Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications?
You'd think they would have solved it by now.
Re: Don't classify, hallucinate
#13Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications? You'd think they would have solved it by now.
Because that's structured data and structured data is usually hidden away from users _and_ machines. Product rarely want to be honest, unless it's B2B in a very competitive market (and even then!). So, yeah, it's not that they are bad, it's that there are few good sources of information.
(Lets ignore for now that no one seems to agree to what should be the spec sheets)
Re: Don't classify, hallucinate
#14Did you validate this by running a A/B test? Main question is were you able to classify back into your known categories correctly all the time, or did the errors compound from the llm hallucination plus embedding search
Using a Nano model, a tad worse than shipping a vocabulary to a larger OpenAI model. (And it’s an huge improvement on not classifying the queries at all). But no classification is perfect. In search in particular, you will also want to have places for manual intervention for high priority queries.
[flagged]
Re: Don't classify, hallucinate
#15Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.
They are already solving the problem with search engines, they're just using an LLM as a first pass to create better embeddings to run a similarity match on first. The difference in latency is likely made up for in accuracy.
Re: Don't classify, hallucinate
#16Re: Don't classify, hallucinate
#17Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.
[dead]
Re: Don't classify, hallucinate
#18Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications? You'd think they would have solved it by now.
Because that's structured data and structured data is usually hidden away from users _and_ machines. Product rarely want to be honest, unless it's B2B in a very competitive market (and even then!). So, yeah, it's not that they are bad, it's that there are few good sources of information. (Lets ignore for now that no one seems to agree to what should be the spec sheets)
An LLM can read websites, right? And turn them into structured data.
Re: Don't classify, hallucinate
#19Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications? You'd think they would have solved it by now.
Because the data, in general, is not included in the LLM model and it needs to search/browse for external information. It cannot look indefinitely so it get the top results from lists, not "evrything".
Re: Don't classify, hallucinate
#20Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications? You'd think they would have solved it by now.
Because that's structured data and structured data is usually hidden away from users _and_ machines. Product rarely want to be honest, unless it's B2B in a very competitive market (and even then!). So, yeah, it's not that they are bad, it's that there are few good sources of information. (Lets ignore for now that no one seems to agree to what should be the spec sheets)
With agentic commerce protocol / unified commerce protocol open ai and gemini are trying to solve this problem.
The idea is to make structured queries using these protocols which can be used to fetch top products matching the user needs instead of just relying on semantic search.
https://developers.openai.com/commerce/specs/file-upload/pro...