This is basically HyDE (Hypothetical Document Embeddings), no? I had tried this approach in the past, worked with limited success.
https://arxiv.org/abs/2212.10496 for others like me hearing about HyDE for the first time
Don't classify, hallucinate
71–80 of 118 posts
Re: Don't classify, hallucinate
#72I can’t believe programming is now at the stage where advice like "first have the computer give you totally wrong answers, then just find a function that maps the wrong answers to the correct ones!" is a thing.
> On two occasions I have been asked, – "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
He clearly didn't know enough about vector embeddings.
Re: Don't classify, hallucinate
#73New embedding models support queries, so you don’t need to hallucinate a document before finding the nearest neighbor. Curious how it compares to this approach since you’d get to skip the LLM altogether.
What does it mean that an "embedding model supports queries"? An embedding model maps text to embedding vectors. You can always perform queries with such embedding vectors against a stored set of embeddings.
Re: Don't classify, hallucinate
#74I can’t believe programming is now at the stage where advice like "first have the computer give you totally wrong answers, then just find a function that maps the wrong answers to the correct ones!" is a thing.
This been the case for a long time!
The entire problem of search is that the user has the wrong data and wants to use it to receive the correct data. That was the start, not the state we’ve ended up at - it is unironically how we got to LLMs.
Re: Don't classify, hallucinate
#75I can’t believe programming is now at the stage where advice like "first have the computer give you totally wrong answers, then just find a function that maps the wrong answers to the correct ones!" is a thing.
> On two occasions I have been asked, – "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. He clearly didn't know enough about vector embeddings.
Unironically, LLM is absolutely amazing at giving you the right answers when you put in wrong input, compared to every other algorithm ever invented.
Re: Don't classify, hallucinate
#76I don't know the exact syntax any more, but I expect this could be solved by a single sql query that uses "inexact but close" queries and a bunch of indexes (and perhaps tags) on each category.
Re: Don't classify, hallucinate
#77Re: Don't classify, hallucinate
#78I can’t believe programming is now at the stage where advice like "first have the computer give you totally wrong answers, then just find a function that maps the wrong answers to the correct ones!" is a thing.
If information is totally wrong then all you have to do is invert it to get the truth. What was it that Sherlock Holmes said? The problem ends up being that it often takes a tremendous number of counterexamples to eliminate everything that is impossible.
Worse is when you don't know whether the answers you have are totally wrong.
Re: Don't classify, hallucinate
#79Nice trick. Couldn't you embed the query though, compare it to the embedding of the categories, then ship only categories that are close to it in the prompt to a smaller model?
It might be a little worse, but it will definitely be way cheaper.
Re: Don't classify, hallucinate
#80I can’t believe programming is now at the stage where advice like "first have the computer give you totally wrong answers, then just find a function that maps the wrong answers to the correct ones!" is a thing.
Eh? “Generate an approximation and refine it algorithmically” is a well-known technique.