Live data from Hacker News

“The world's most private search engine”

ixquick.com

1–10 of 22 posts

Re: “The world's most private search engine”

#3

Does that mean it searches private info? Or does it mean that nobody uses it? Maybe it means that the searches are anonymized?

The last. It basically acts as a proxy: You type in your query, and IxQuick (aka startpage.com) queries Google on your behalf, thereby shielding you from the associated analytics and tracking. It also claims to not store the IP address associated with an individual query.

Edit: more info here: https://ixquick.com/eng/aboutixquick/

Re: “The world's most private search engine”

#4

Does that mean it searches private info? Or does it mean that nobody uses it? Maybe it means that the searches are anonymized?

From their site:

-You have a right to privacy.

-Your search data should never fall into the wrong hands.

-The only real solution is quickly deleting your data or not storing them to begin with.

-Since January 2009 we do not record our users' IP addresses anymore.

-Your personal data are not shared with any third party.

-We are the first and only search engine to do so.

-Since then we have added many other features that protect your privacy.

-Our initiative is receiving an overwhelmingly positive response!

https://www.ixquick.com/eng/protect-privacy.html

Re: “The world's most private search engine”

#5
I usually use DuckDuckGo.com, because they have a neat toolbar and I want to support their cause, but everytime I don't get the results I want through DDG (which happens rarely) I just type !sp + searchterm into my search bar and it searches via startpage.com, which uses googles search results without my IP address or searches being recorded, no identifying or tracking cookies are used and SSL encryption is set by default.

Startpage.com was launched in 2009 by ixquick to create the same service via a URL that is both easier to remember and spell.

Re: “The world's most private search engine”

#7
I wonder what it takes to host a full search engine for the Internet now, if an organization wanted genuinely private search. Presumably you can buy crawls from somewhere, but does anyone sell a self-hostable general web search engine?

Re: “The world's most private search engine”

#8
I've noticed while using this that the results are often very different to google's (and generally inferior).

Sometimes I find myself switching back to "regular" google, but not as often as when I used duck duck go.

One thing that really bugs me is that I can't do currency conversions or calculations in this one.

Re: “The world's most private search engine”

#9
post #7

I wonder what it takes to host a full search engine for the Internet now, if an organization wanted genuinely private search. Presumably you can buy crawls from somewhere, but does anyone sell a self-hostable general web search engine?

There are GPL'ed peer to peer search engines like YaCy.

https://en.wikipedia.org/wiki/YaCy

Re: “The world's most private search engine”

#10
post #7

I wonder what it takes to host a full search engine for the Internet now, if an organization wanted genuinely private search. Presumably you can buy crawls from somewhere, but does anyone sell a self-hostable general web search engine?

That I'm aware of there are only three places to get actual search index results in the US, Google, Bing (via Yahoo BOSS), and of course Blekko. You can also get index results from both Yandex (Russian) and Baidu (Chinese) but both serve them out of data centers in either Amsterdam or Asia respectively so latency is a bit longer.

With modern hardware (especially 3TB drives) you can crawl a significant chunk of the Internet with as few as a hundred machines but in practice you will want closer to 300. Of course crawling is but one (relatively small) step in the series of steps needed to create an index, next up is page extraction, then term extraction, then semantic analysis followed by meta-data decoration (is it likely porn or maybe malware or junk, did it change since the last time you looked at it, what is a good snippet, etc etc) Then you process link information and perhaps click stream data and create your ranking model, which then lets you build an index. At which point you need another bunch of machines which can efficiently take a query string and put together a set of documents that might be a good fit (straightforward for 1 term, harder for 2 terms, exponentially worse for 3, 4, 5, 6 or more terms).

Depending on latency requirements and load you make that part of the system in another few hundred machines.

Search results though also have an inverse power law associated with them, the most popular consumer results fit in a .5 billion URL index, the 95th percentile in a 3 - 5 billion URL index, the 99th percentile is probably closer to 45 - 50 billion URL index. At the 95th percentile you're up to a few thousand machines to hold it all.

It isn't really possible to build on a generic VPS yet (like ECS on steroids) without having latency being ridiculous. It would also be expensive, the last time I did the calculation for the memory/compute/storage resources (pretending the latency could be fixed) it was about $2.5M/month in resource fees for a 1/2 billion page index and crawl. Since that time both Amazon and Google have lowered some prices so it may be more affordable now.

Putting it in a data center is much cheaper than that but not quite cheap enough to just let it sit around without monetizing it somewhat.

I'm a bit surprised that someone hasn't put something together, there are bits and pieces out there. But even self-hosted you're looking at least a few hundred thousand $ a month to keep it up and since Yahoo will sell you access to their search API for a couple of bucks per thousand queries it's probably not worth it for most people to try and run their own.

Post reply on HN