61–70 of 89 posts
It also needs a SearXNG server to run, so I recently pivoted towards a self-contained solution: https://github.com/denysvitali/search-mcp
I won't say searxng can help you degoogle because metasearch engine calls other search engines, e.g., google, to collect results. However, if you try searxng, you can at least get rid of things like ai reviews in no time.
In the end, thank you people after searxng project and public instances.
SearXNG is my daily internet search now +5 years; with YaCY Backends and else as fallback. I also build internal document search or RAG applications with this setup (SearXNG also support json results). However, there are some downer I accept because of privacy: 1. Its slower and the results are not that good then with others. But fast and good enough for most of my queries. 2. From time to time you get blocked on the…
Same here
> with YaCY Backends and else as fallback.
Do you run your own "super fast" YaCy instance? or with specific settings?
My experience with YaCy is it doesn't fit in the backend of SearX since YaCy kind of slowly stream results for about 30 seconds...
I also have a local `kiwix-serve` serving ZIM files of wikipedia, wiktionary, gutemberg, archwiki, etc. but same problem the kiwix search engine [0] doesn't really fit as a backend for SearX as it returns too many results and pollute the SearX result page.
What I haven't done yet is trying to plug SearX to a local Recoll instance [1]. But Recoll doesn't support indexing ZIM files... but could be useful for other archived html documents.
I would be curious to know more about a working setup since search is hard to get right.
- [0] https://kiwix-tools.readthedocs.io/en/latest/kiwix-serve.htm...
- [1] https://docs.searxng.org/dev/engines/online/recoll.html
Ohi, I'm the original creator of Searx, but due to the limitations of the metasearch concept I'm not involved in the development anymore. My new search project is https://github.com/asciimoo/hister ( https://hister.org/ ). Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. Storing full page content allows serving offline result previews…
Earlier quoted context omitted.
I'm sorry for not taking the time to read the docs, but I have a question. Some 20 years ago a friend of mine has set up a local proxy (python if I'm not mistaken) that was gathering all his web traffic and served him as a long term memory. The proxy had a web interface and allowed him to quickly find something he saw ca. 10 days ago, or that specific algorithm he recalls but can't remember it's name. For years I've…
Absolutely, this is a great example where Hister can shine. I started Hister as a proxy as well, but quickly switched to the current extension based approach, because intercepting HTTPS traffic requires a MiTM proxy which is much more painful to setup than installing a browser extension.
Now for many of us the browser extension approach is not possible (mobile usage, security, etc.)
My feeling is for a lot of users there is really a third way apart from the MiTM proxy or Browser extension approach. I actually do not want my "personal" / "logged in" pages to be indexed. This is a bit like the MS recall nightmare (self hosted version).
Any way to get the list of URL visited (with something like Privoxy, or maybe one of those popular ad blockers like Pi Hole but I guess they just get DNS queries?) and then importing it with some filtering rules with a nightly batch job is good enough for a lot of people.
The browser import [1] is great but I guess hard to use with mobile...
- [0] https://www.privoxy.org/
Ohi, I'm the original creator of Searx, but due to the limitations of the metasearch concept I'm not involved in the development anymore. My new search project is https://github.com/asciimoo/hister ( https://hister.org/ ). Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. Storing full page content allows serving offline result previews…
YaCY has a proxy mode that automatically index your web-serving. In my experience, the index grow in size very fast and reaches ~100GB or more. How does the index size of Hister compare to that?
I'm using it for 6-7 months and my index size is below 1GB with almost 10k pages.
Also, a downside of the proxy approach: it does not handle properly JS based websites and cannot identify dynamic content changes. Our extension periodically checks if the browser tabs' content has been changed and automatically updates the index when change detected.
Ohi, I'm the original creator of Searx, but due to the limitations of the metasearch concept I'm not involved in the development anymore. My new search project is https://github.com/asciimoo/hister ( https://hister.org/ ). Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. Storing full page content allows serving offline result previews…
I was considering paying someone to build something like this at some point. With two jobs, I eventually had no time to even organize what I find. It's just piles of links in text files. Can I give your software a huge list of URL's to index? Or do I need to use browser automation to open them a few at a time with it caching and indexing them?
Hister has a built in crawler with standard HTTP lib and browser based backends, you can feed your link collection to it. Also, Hister supports importing your existing browser history automatically using either of the mentioned backends.
Earlier quoted context omitted.
Absolutely, this is a great example where Hister can shine. I started Hister as a proxy as well, but quickly switched to the current extension based approach, because intercepting HTTPS traffic requires a MiTM proxy which is much more painful to setup than installing a browser extension.
I found Hister a few month ago and was amazed by it. Now for many of us the browser extension approach is not possible (mobile usage, security, etc.) My feeling is for a lot of users there is really a third way apart from the MiTM proxy or Browser extension approach. I actually do not want my "personal" / "logged in" pages to be indexed. This is a bit like the MS recall nightmare (self hosted version). Any way to get…
There is already an ongoing discussion about the topic: https://github.com/asciimoo/hister/issues/387
The currently discussed solution relies on the browser extension, but mobile Firefox has extension support.
Ohi, I'm the original creator of Searx, but due to the limitations of the metasearch concept I'm not involved in the development anymore. My new search project is https://github.com/asciimoo/hister ( https://hister.org/ ). Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. Storing full page content allows serving offline result previews…