Now if someone can hook this into Plandex (also shared today - https://news.ycombinator.com/item?id=39918500) to make a tool that enables you to collaborate with AI without any of your code leaving your computer, that would be amazing!
Show HN: I've built a locally running Perplexity clone
21–30 of 140 posts
Re: Show HN: I've built a locally running Perplexity clone
#22Earlier quoted context omitted.
Sorry to repeat the same question I just asked the other commenter in this thread, but could you link the model page and recommend a specific level of quantization for the models you've referenced? I'd love to play with these models and see what you're talking about.
It's from nous research https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B Q5 is minimum.
Thank you — from that page, at the bottom, I was able to find this link to what I think are the quantized versions
https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-...
If you have the time, could you explain what you mean by "Q5 is minimum"? Did you determine that by trying the different models and finding this one is best, or did someone else do that evaluation, or is that just generally accepted knowledge? Sorry, I find this whole ecosystem quite confusing still, but I'm very new and that's not your problem.
Re: Show HN: I've built a locally running Perplexity clone
#23The big secret about perplexity is they havent done much beyond using off the shelf models
Re: Show HN: I've built a locally running Perplexity clone
#24This is cool, haven't run this yet but seems really promising. Am thinking how this can be a super useful to hook with internal corporate search engines and then get answers from that.
Good to see more of these non API key products being built (connected to local llms)
Re: Show HN: I've built a locally running Perplexity clone
#25Impressive, I don't think I've seen a local model call upon specialised modules yet (although I can't keep up with everything going on). I too use local 7b open-hermes and it's really good.
Have you looked into tools like CrewAI [0]?
Re: Show HN: I've built a locally running Perplexity clone
#26Happy to answer any questions and open for suggestions :) It's basically a LLMs with access to a search engine and the ability to query a vector db. The top n results from each search query (initialized by the LLM) will be scraped, split into little chunks and saved to the vector db. The LLM can then query this vector db to get the relevant chunks. This obviously isn't as comprehensive as having a 128k context LLM ju…
Your project looks very cool. I had on my ‘list’ to re-learn Typescript (I took a TS course about 5 years ago, but didn’t do anything with it) so I just cloned your repo so I can experiment with it.
EDIT: I just noticed that most of the code is Go. Still going to play with it!
Re: Show HN: I've built a locally running Perplexity clone
#27Perplexity seems to be a chatbot competitor.
Re: Show HN: I've built a locally running Perplexity clone
#28The big secret about perplexity is they havent done much beyond using off the shelf models
How did they secure funds in that case?
Re: Show HN: I've built a locally running Perplexity clone
#29The big secret about perplexity is they havent done much beyond using off the shelf models
How did they secure funds in that case?
That is probably exactly why they got funding. You can sell it as focus on adding new features and leveraging the best available tools before reinventing the wheel.
They do train their own models now, but for about a year they just forwarded calls to models like gpt3.5T. You still have the option to use models not trained by perplexity.
Re: Show HN: I've built a locally running Perplexity clone
#30Earlier quoted context omitted.
How did they secure funds in that case?
That is probably exactly why they got funding. You can sell it as focus on adding new features and leveraging the best available tools before reinventing the wheel. They do train their own models now, but for about a year they just forwarded calls to models like gpt3.5T. You still have the option to use models not trained by perplexity.
which is why their engagement and model responses suck. the other competitors are far better
C.ai and Pi comes to mind