SearX: Privacy-respecting metasearch engine
1–10 of 30 posts
Re: SearX: Privacy-respecting metasearch engine
#2This is awesome. I have been wanting to build something like this for a while but never had the time.
Re: SearX: Privacy-respecting metasearch engine
#3Even with a pool of proxies, I would expect an instance of this "metasearch engine" to quickly get banned by the other search engines. The same IP running thousands of queries and scraping its content (which is against their ToS) should be easily detectable.
Re: SearX: Privacy-respecting metasearch engine
#4Even with a pool of proxies, I would expect an instance of this "metasearch engine" to quickly get banned by the other search engines. The same IP running thousands of queries and scraping its content (which is against their ToS) should be easily detectable.
This is self-hosted, so I'm assuming it's running under the assumption that each person hosts their own instance and uses that instance. The number of queries coming from the instance in that case wouldn't look too out of the ordinary.
Re: SearX: Privacy-respecting metasearch engine
#5Even with a pool of proxies, I would expect an instance of this "metasearch engine" to quickly get banned by the other search engines. The same IP running thousands of queries and scraping its content (which is against their ToS) should be easily detectable.
Building on that issue, I'd like to add that it would be nice to have a feature that alerts a user that certain a search engine is denying requests. It's visible in the logs or settings somewhere, but usually I find myself wondering for a while why my search queries aren't accurate before heading off to figure out why.
Still a great project though, I use it every day.
Re: SearX: Privacy-respecting metasearch engine
#6This is awesome. I have been wanting to build something like this for a while but never had the time.
There's also pears search, which died sometime after being funded by Mozilla. I don't think it was malicious intent, but no one can explain why the project is inactive.
Re: SearX: Privacy-respecting metasearch engine
#7On a related note, have any of you tried FindX?
https://github.com/privacore/open-source-search-engine
Looks promising but haven't used it much yet.
Re: SearX: Privacy-respecting metasearch engine
#8Even with a pool of proxies, I would expect an instance of this "metasearch engine" to quickly get banned by the other search engines. The same IP running thousands of queries and scraping its content (which is against their ToS) should be easily detectable.
This is self-hosted, so I'm assuming it's running under the assumption that each person hosts their own instance and uses that instance. The number of queries coming from the instance in that case wouldn't look too out of the ordinary.
Then that defeats the purpose of trying to be privacy focused if your search queries aren't mixed with other people's queries.
Re: SearX: Privacy-respecting metasearch engine
#9On a related note, have any of you tried FindX? https://www.findx.com/ https://github.com/privacore/open-source-search-engine Looks promising but haven't used it much yet.
I like this part "it draws its results from its own bot that crawls the web". There aren't too many that use their own bot.
Tsignal (https://deepsearch.tsignal.io/) is another that uses it's own bot with a little AI tossed into the mix. And... it's currently not accessible:(
Wanted to add that I'm currently on Opera and using an extension named Search All[0]. After conducting a search via your default search engine, this extension places a bar with a list of user configurable search engines. This allows you to search alternatives easily using the same keywords.
One great feature is that if you go directly to a search engine, click on the Search All icon, it almost always identifies it with correct parameters and can be easily added. Just added findX to my bar (for testing).
I plan on going back to Firefox and wish FF had something like this (part of the reason I'm posting this).
[0] https://addons.opera.com/en/extensions/details/search-all/?d...
Re: SearX: Privacy-respecting metasearch engine
#10Even with a pool of proxies, I would expect an instance of this "metasearch engine" to quickly get banned by the other search engines. The same IP running thousands of queries and scraping its content (which is against their ToS) should be easily detectable.
I've been running multiple SearX instances for goodness knows how long and this has never happened to me. I'm not aware of this happening either.
SearX uses multiple sources for queries, so you'd have to be banned by quite a few search engines to stop it being useful.
Also relevant is filtron[1], an application firewall built-in to SearX that rate limits searches.