I if a quick poke through the source and it seems like there’s not much reason this couldn’t run on macOS? It seems that ollama is doing the inference and then there’s a go binary doing everything else? I might give it a go and see what happens!
Show HN: I've built a locally running Perplexity clone
71–80 of 140 posts
Re: Show HN: I've built a locally running Perplexity clone
#72Earlier quoted context omitted.
Would it not be possible to create a search engine that only crawls certain sites?
I was most interested in the offline aspect of it, which I wouldn't know where to even start with if I were to fork. How do you parse and efficiently store large, unstructured information for arbitrary, unstructured queries?
You put it in a search server, like ElasticSearch or Meili.
Re: Show HN: I've built a locally running Perplexity clone
#73Earlier quoted context omitted.
It's from nous research https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B Q5 is minimum.
Thank you — from that page, at the bottom, I was able to find this link to what I think are the quantized versions https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-... If you have the time, could you explain what you mean by "Q5 is minimum"? Did you determine that by trying the different models and finding this one is best, or did someone else do that evaluation, or is that just generally accepted knowledg…
Talking GGUF, Usually the higher you can afford to go wrt. quantization(e.g. Q5 is better than Q4, etc), the better. A Q6_K has minimal performance loss from the Q8, so in most cases if you can fit a Q6_K it's recommended to just use that. TheBloke's READMEs[0] usually have a good table summarizing each quantization level.
If you're RAM constrained, you'll also have to make trade-offs about the context length. e.g. you could have 8 GB RAM and a Q5 quant with shorter context, vs Q3 with longer, etc.
Re: Show HN: I've built a locally running Perplexity clone
#74Happy to answer any questions and open for suggestions :) It's basically a LLMs with access to a search engine and the ability to query a vector db. The top n results from each search query (initialized by the LLM) will be scraped, split into little chunks and saved to the vector db. The LLM can then query this vector db to get the relevant chunks. This obviously isn't as comprehensive as having a 128k context LLM ju…
Wonderful work! is it possible to make it only use a subset of the web? (Only sites that I trust and think are relevant to producing an accurate answer), and are there ways to possibly make it work offline on pre installed websites? (wikipedia, some other wikis and possibly news sites that are archived locally), and how about other forms of documents? (books and research papers as pdfs)
uhhhh both ideas are great, would you like to turn them into github issues? i will definitely look into both of them :)
Re: Show HN: I've built a locally running Perplexity clone
#75Earlier quoted context omitted.
Wonderful work! is it possible to make it only use a subset of the web? (Only sites that I trust and think are relevant to producing an accurate answer), and are there ways to possibly make it work offline on pre installed websites? (wikipedia, some other wikis and possibly news sites that are archived locally), and how about other forms of documents? (books and research papers as pdfs)
Llocalsearch uses searxng which has a feature to blacklist/whitelist sites for various purposes.
also a great idea to expose this to the frontend. thanks :)
Re: Show HN: I've built a locally running Perplexity clone
#76Exciting project. Trying to install it but running into some issues with searxng. Anyone else?
Re: Show HN: I've built a locally running Perplexity clone
#77Earlier quoted context omitted.
If you're open to it, it would be great if you could make a post explaining how you built this. Even if it's brief. Trying to learn more about this space and this looks pretty cool. And ofc, nice work!
a primer - https://github.com/nilsherzig/LLocalSearch/issues/17
guys, i didn't thought there would be this much interest in my project haha. I feel kinda bad for just posting it in this state haha. I would love to make a more detailed post on how it works in the future (keep an eye on the repo?)
Re: Show HN: I've built a locally running Perplexity clone
#78Exciting project. Trying to install it but running into some issues with searxng. Anyone else?
please tell me about the problem or open an issue :)
Re: Show HN: I've built a locally running Perplexity clone
#79Earlier quoted context omitted.
Thanks :). It's just a lot of prompting and string parsing. There are models like "Hermes-2-Pro-Mistral" (the one from the video) which are trained to work with function signatures and outputting structured text. But at the end it's just strings in > strings out, haha. But its fun (and sometimes frustrating) to use LLMs for flow control (conditions, loops...) inside your programs.
Got a link for that one? I have found a few with Hermes-2-Mistral in the name.
[deleted]
Re: Show HN: I've built a locally running Perplexity clone
#80This is so cool! And the fact that you can use Ollama as 'llm backend' makes it sustainable. didn't see how to switch models in the demo, that might be worth to highlight in readme..
I have a 'feature request', can we manage which sites are being used by some categories in the frontend? For example, if I build a list of websites and out them under "coding", then I'd like to use those to answer my programming questions. Meanwhile, I'd like to add an "art" category for museum's homepages so that I can ask which year was XYZ painting from. And so on. The current implementation looks like the inter-operability with searing is more static... IDK if searxng has an API to switch those filters or if they can be managed already through 'profiles'.. that kind of thing..