Show HN: I've built a locally running Perplexity clone • Hacker News Board

Re: Show HN: I've built a locally running Perplexity clone

#41

Post by BOOSTERHIDROGEN » Thu, Apr 04, 2024, 5:59 AM UTC

peter_l_downs wrote: post #22

BOOSTERHIDROGEN wrote: post #20

Earlier quoted context omitted.

It's from nous research https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B Q5 is minimum.

Thank you — from that page, at the bottom, I was able to find this link to what I think are the quantized versions https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-... If you have the time, could you explain what you mean by "Q5 is minimum"? Did you determine that by trying the different models and finding this one is best, or did someone else do that evaluation, or is that just generally accepted knowledg…

It's the best balance if you have limited compute performance.

Re: Show HN: I've built a locally running Perplexity clone

#42

Post by bobby_the_whale » Thu, Apr 04, 2024, 6:02 AM UTC

KuriousCat wrote: post #36

basbuller wrote: post #29

Earlier quoted context omitted.

That is probably exactly why they got funding. You can sell it as focus on adding new features and leveraging the best available tools before reinventing the wheel. They do train their own models now, but for about a year they just forwarded calls to models like gpt3.5T. You still have the option to use models not trained by perplexity.

I still don't get it. What was the USP here? What is the allure in it for the investors?

[dead]

Re: Show HN: I've built a locally running Perplexity clone

#43

Post by bobby_the_whale » Thu, Apr 04, 2024, 6:06 AM UTC

dcreater wrote: post #40

nilsherzig wrote: post #11

Earlier quoted context omitted.

Thanks :) Die LLM makes these decisions on its own. If it writes a message which contains a tool call (Action: Web search Action Input: weight of a llama) the matching function will be executed and the response returned to the LLM. It's basically chatting with the tool. You can toggle the log viewer on the top right, to get more detail on what it's doing and what is taking time. Timing depends on multiple things: - t…

> Die LLM You mean the? The German is bleeding through haha

[dead]

Re: Show HN: I've built a locally running Perplexity clone

#44

Post by rzzzt » Thu, Apr 04, 2024, 6:13 AM UTC

dcreater wrote: post #40

nilsherzig wrote: post #11

Earlier quoted context omitted.

Thanks :) Die LLM makes these decisions on its own. If it writes a message which contains a tool call (Action: Web search Action Input: weight of a llama) the matching function will be executed and the response returned to the LLM. It's basically chatting with the tool. You can toggle the log viewer on the top right, to get more detail on what it's doing and what is taking time. Timing depends on multiple things: - t…

> Die LLM You mean the? The German is bleeding through haha

Wolfenstein 3D did it first! And then The Simpsons as well.

Re: Show HN: I've built a locally running Perplexity clone

#45

Post by visarga » Thu, Apr 04, 2024, 6:24 AM UTC

mmahemoff wrote: post #37

nilsherzig wrote: post #19

Earlier quoted context omitted.

searxng, which is a locally running meta search engine combining a lot of different sources (including Google and co)

This might be more of a searxng question, but doesn't it quickly run up against anti-bot measures? CAPTCHA challenges and Forbidden responses? I can see the manual has some support for dealing with CAPTCHA [1], but in practical terms, I would guess a tool like this can't be used extensively all day long. I'm wondering if there's a search API that would make the backend seamless for something like this. 1. https://doc…

As a last resort we could have AI work on top of a real web browser and solving captchas as well. Should look like normal usage. I think these kinds of systems LLM + RAG + Web Agent will become widespread and the preferred method to interact with the web.

We can escape all ads and dark UI patterns by delegating this task to AI agents. We could have it collect our feeds, filter, rank and summarize them to our preferences, not theirs. I think every web browser, operating system and mobile device will come equipped with its own LLM agent.

The development of AI screen agents will probably get a big boost from training on millions of screen capture videos with commentary on YouTube. They will become a major point of competition on features. Not just browser, but also OS, device and even the chips inside are going to be tailored for AI agents running locally.

Re: Show HN: I've built a locally running Perplexity clone

#46

Post by hubraumhugo » Thu, Apr 04, 2024, 7:10 AM UTC

Excellent work! Cool side projects like that will eventually help you get hired by a top startup or may even lead to building your own.

I can only encourage other makers to post their projects on HN and put them out into the world.

Re: Show HN: I've built a locally running Perplexity clone

#47

Post by ml-anon » Thu, Apr 04, 2024, 7:14 AM UTC

Did you really make a perplexity clone if you didn’t spend more time promoting yourself on Twitter and LinkedIn than on the engineering?

Re: Show HN: I've built a locally running Perplexity clone

#48

Post by arflikedog » Thu, Apr 04, 2024, 7:19 AM UTC

A while back you commented on my personal project Airdraw which I really appreciated. This looks awesome and you're well on your way to another banger project - looking forward to toying around with this :)

Re: Show HN: I've built a locally running Perplexity clone

#49

Post by keefle » Thu, Apr 04, 2024, 7:25 AM UTC

nilsherzig wrote: post #3

Happy to answer any questions and open for suggestions :) It's basically a LLMs with access to a search engine and the ability to query a vector db. The top n results from each search query (initialized by the LLM) will be scraped, split into little chunks and saved to the vector db. The LLM can then query this vector db to get the relevant chunks. This obviously isn't as comprehensive as having a 128k context LLM ju…

Wonderful work!

is it possible to make it only use a subset of the web? (Only sites that I trust and think are relevant to producing an accurate answer), and are there ways to possibly make it work offline on pre installed websites? (wikipedia, some other wikis and possibly news sites that are archived locally), and how about other forms of documents? (books and research papers as pdfs)

Re: Show HN: I've built a locally running Perplexity clone

#50

Post by manojlds » Thu, Apr 04, 2024, 7:31 AM UTC

visarga wrote: post #45

mmahemoff wrote: post #37

Earlier quoted context omitted.

This might be more of a searxng question, but doesn't it quickly run up against anti-bot measures? CAPTCHA challenges and Forbidden responses? I can see the manual has some support for dealing with CAPTCHA [1], but in practical terms, I would guess a tool like this can't be used extensively all day long. I'm wondering if there's a search API that would make the backend seamless for something like this. 1. https://doc…

As a last resort we could have AI work on top of a real web browser and solving captchas as well. Should look like normal usage. I think these kinds of systems LLM + RAG + Web Agent will become widespread and the preferred method to interact with the web. We can escape all ads and dark UI patterns by delegating this task to AI agents. We could have it collect our feeds, filter, rank and summarize them to our preferen…

If everyone consumes like that what's even the incentive for content creators?