For web search, also consider the Kagi and Tavily APIs.
Local Deep Research – ArXiv, wiki and other searches included
11–20 of 35 posts
Re: Local Deep Research – ArXiv, wiki and other searches included
#12Looks very cool. How does this compare to the RAG features provided by open-webui?
There is web search and a way to embed documents, but so far it seems like the results are subpar as details are lost in embeddings. Is this much better?
Re: Local Deep Research – ArXiv, wiki and other searches included
#13For web search, also consider the Kagi and Tavily APIs.
Thank you I will add
Re: Local Deep Research – ArXiv, wiki and other searches included
#14I think the guy who’ll make the 3D game-like GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared, current chatbots are a bit like a command line in comparison. I just started ASK HN to let people and me share their AI safety ideas, both crazy and not: https://news.ycombinator.com/item?id=43332593
Re: Local Deep Research – ArXiv, wiki and other searches included
#15Looks very cool. How does this compare to the RAG features provided by open-webui? There is web search and a way to embed documents, but so far it seems like the results are subpar as details are lost in embeddings. Is this much better?
Give me a question and I can give you the output? So you can compare.
Re: Local Deep Research – ArXiv, wiki and other searches included
#16This is cool!
If you want to add embeddings over internet as a source, you should try out exa.ai. Includes: wikipedia, tens of thousands of news feeds, Github, 70M+ papers including all of arxiv, etc.
disclaimer: I am one of the founders (:
Re: Local Deep Research – ArXiv, wiki and other searches included
#17I applaud the effort for the local (lo-fi) space ! Yet, reading over the example linked in the docs (which does not seem cheery-picked, kudos for that!), my impression is that the document is a rather messy outcome [1]. I think what's missing is one (or more) step in-between, possible a graph database (eg[2]), which the LLM can place all it's information in, see relevant interconnections, query to question itself, an…
I had my own spin on deep research which you might find this easier to navigate: https://github.com/btahir/open-deep-research
Re: Local Deep Research – ArXiv, wiki and other searches included
#18Looks very cool. How does this compare to the RAG features provided by open-webui? There is web search and a way to embed documents, but so far it seems like the results are subpar as details are lost in embeddings. Is this much better?
Give me a question and I can give you the output? So you can compare.
I tried it myself. It looks like this can do a lot more than open-webui's web search in terms of detail, which sounds useful, thanks for making it open source.
It seems to have a weird behavior of specifying a date when I didn't ask for it, is this expected? Also, I feel like searching "questions" is not optimal for most search engines, and it should instead search in terms of keywords.
Also, I wish there can be a more informative log at a slightly higher level - I don't need to see every request being made, but I do want to see a summary of what's happening at each step, like the prompt used, result, and the new search being done.
On another note, for local models, reasoning models have significant advantages over non-reasoning models. Can they be used for this?
Re: Local Deep Research – ArXiv, wiki and other searches included
#19I think the guy who’ll make the 3D game-like GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared, current chatbots are a bit like a command line in comparison. I just started ASK HN to let people and me share their AI safety ideas, both crazy and not:…
You just posted the same comment three times in three different posts in 10 minutes. I'd say it would be nice to take it a bit slower...
Re: Local Deep Research – ArXiv, wiki and other searches included
#20I think the guy who’ll make the 3D game-like GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared, current chatbots are a bit like a command line in comparison. I just started ASK HN to let people and me share their AI safety ideas, both crazy and not:…
You just posted the same comment three times in three different posts in 10 minutes. I'd say it would be nice to take it a bit slower...
Yep, it’s a bit different, I won’t do it again. The problem is important, I wanted to hear other people’s ideas, you can google “share AI safety ideas”, I posted the same question in a bunch of places and it created some discussions