what does this have to do with Perplexity? it should reference the underlying models used instead
Show HN: I've built a locally running Perplexity clone
31–40 of 140 posts
Re: Show HN: I've built a locally running Perplexity clone
#32The big secret about perplexity is they havent done much beyond using off the shelf models
I've been working on a small personal project similar to this and agree that replicating the overall experience provided by Perplexity.ai, or even improving it for personal use, isn't that challenging.
(The concerns of scale or cost are less significant in personal projects. Perplexity doesn't do too much planning or query expansion, nor does it dig super deep into the sources afaik)
I must say, though, that they are doing a commendable job integrating sources like YouTube and Reddit. These platforms benefit from special preprocessing and indeed add value.
Re: Show HN: I've built a locally running Perplexity clone
#33what does this have to do with Perplexity? it should reference the underlying models used instead
The UX is comparable.
Re: Show HN: I've built a locally running Perplexity clone
#34Happy to answer any questions and open for suggestions :) It's basically a LLMs with access to a search engine and the ability to query a vector db. The top n results from each search query (initialized by the LLM) will be scraped, split into little chunks and saved to the vector db. The LLM can then query this vector db to get the relevant chunks. This obviously isn't as comprehensive as having a 128k context LLM ju…
"normal consumer GPU"... well mine is a 4GB 6600.. so I guess that varies.
Re: Show HN: I've built a locally running Perplexity clone
#35Earlier quoted context omitted.
That is probably exactly why they got funding. You can sell it as focus on adding new features and leveraging the best available tools before reinventing the wheel. They do train their own models now, but for about a year they just forwarded calls to models like gpt3.5T. You still have the option to use models not trained by perplexity.
which is why their engagement and model responses suck. the other competitors are far better C.ai and Pi comes to mind
Wait, are you directly comparing Perplexity and C.ai or Pi? Perplexity is a search engine, Pi is a chatbot, and C.ai is roleplay? Their value propositions are very different
Re: Show HN: I've built a locally running Perplexity clone
#36Earlier quoted context omitted.
How did they secure funds in that case?
That is probably exactly why they got funding. You can sell it as focus on adding new features and leveraging the best available tools before reinventing the wheel. They do train their own models now, but for about a year they just forwarded calls to models like gpt3.5T. You still have the option to use models not trained by perplexity.
I still don't get it. What was the USP here? What is the allure in it for the investors?
Re: Show HN: I've built a locally running Perplexity clone
#37Earlier quoted context omitted.
What is the search engine that it uses?
searxng, which is a locally running meta search engine combining a lot of different sources (including Google and co)
This might be more of a searxng question, but doesn't it quickly run up against anti-bot measures? CAPTCHA challenges and Forbidden responses? I can see the manual has some support for dealing with CAPTCHA [1], but in practical terms, I would guess a tool like this can't be used extensively all day long.
I'm wondering if there's a search API that would make the backend seamless for something like this.
Re: Show HN: I've built a locally running Perplexity clone
#38Happy to answer any questions and open for suggestions :) It's basically a LLMs with access to a search engine and the ability to query a vector db. The top n results from each search query (initialized by the LLM) will be scraped, split into little chunks and saved to the vector db. The LLM can then query this vector db to get the relevant chunks. This obviously isn't as comprehensive as having a 128k context LLM ju…
If you're open to it, it would be great if you could make a post explaining how you built this. Even if it's brief. Trying to learn more about this space and this looks pretty cool. And ofc, nice work!
Re: Show HN: I've built a locally running Perplexity clone
#39Excellent work! I plan to use it with existing LLMs tbh, but great to see it working locally also! Thank you so much for sharing. I love the architecture.
Re: Show HN: I've built a locally running Perplexity clone
#40This is really neat! I have questions: “Needs tool usage” and “found the answer” blocks in your infra, how are these decisions made? Looking at the demo, it takes a little time to return results, from the search, vector storage and vector db retrieval, which step takes the most time?
Thanks :) Die LLM makes these decisions on its own. If it writes a message which contains a tool call (Action: Web search Action Input: weight of a llama) the matching function will be executed and the response returned to the LLM. It's basically chatting with the tool. You can toggle the log viewer on the top right, to get more detail on what it's doing and what is taking time. Timing depends on multiple things: - t…
> Die LLM
You mean the? The German is bleeding through haha