Live data from Hacker News

Building a Search Engine from Scratch

0x65.dev

111–120 of 151 posts

Re: Building a Search Engine from Scratch

#111

If you’re listening Cliqz, I (and I believe others) would pay $50-100 a month for a search engine that gave me the ability to blacklist sites. I think this alone could solve the biggest problem I have with google which is the reversion to the (what appears to be regressive) mean of the internet today.

That is a good ideas. If we can do it in a privacy-preserving way, which seems doable, we will most likely do it (although we would be constrained by resources available)

Re: Building a Search Engine from Scratch

#112
post #38

Earlier quoted context omitted.

> They should market (and engineer) strongly on privacy since that's where Google is weak. How can you build a privacy oriented search engine and still make money?

I'd pay a subscription fee given that none of my digital fingerprints is being logged.

Marc from Cliqz here: it’s something I’m thinking about a lot since years, it’s a very clean model. We even did some tests, but unfortunately the majority of people is not like hacker news. What is a reasonable price point for you per month? How would you pay (ie would you feel comfortable giving us your credit card and create an account? (I don’t think crypto is big enough yet to solve this)

Re: Building a Search Engine from Scratch

#113
post #99

But is Cliqz a search engine, or is it a web browser with a search bar set to a default search engine (or a Firefox extension that accomplishes the same thing). Cliqz bought Ghostery to acquire a pool of privacy-conscious users. The goal is to show them ads. Not sure how excited they will be about that. If Cliqz really is a search engine, can a user submit a query to the database using her own choice of tcp/http clie…

https://beta.cliqz.com

Or via Tor browser as Onion service: search4tor7txuze.onion (quite unique - if available it will show onion services instead of www domains).

Re: Building a Search Engine from Scratch

#114
post #46
post #29

That Cliqz is trying to actually build a new search stack is commendable. This is way more exciting to me than DuckDuckGo and other services that just package up Bing search results under different branding. I'm skeptical that they'll be successful, but I wish them the best. They should market (and engineer) strongly on privacy since that's where Google is weak.

« DuckDuckGo and other services that just package up Bing search results » Had to look that up. I found https://help.duckduckgo.com/results/sources/ Bing is just one of "hundreds of vertical sources delivering" results to DuckDuckGo.

If I recall, in the very beginning they were massively using Yahoo and to some extent Bing. It might be that over the years Bing has become a more reliable source.

Re: Building a Search Engine from Scratch

#115
post #40
post #38

Earlier quoted context omitted.

> They should market (and engineer) strongly on privacy since that's where Google is weak. How can you build a privacy oriented search engine and still make money?

By selling ads solely against the query, and figuring out a way to track conversions against generated unique referring URL instead of using cookies. Maybe it's less profitable, but it might still be profitable, and it gives you a foothold.

Or a sponsored and unsponsored section. The only thing you pay for is for being higher up in the sponsored pane. No need for tracking. No conversion clicks tracked. Only higher on the sponsored listing.

Re: Building a Search Engine from Scratch

#116

Is the Human Web dataset available for download?

Data release, it's not possible, but if people want to come and do experiments on the data or try to test it for privacy, we are more than welcome to host them. There is no formal process in any way, best effort, we have done several times in the past. If you are very interested contact us and we will see if we can accommodate you.

[Disclaimer: I work at Cliqz]

Re: Building a Search Engine from Scratch

#117

Earlier quoted context omitted.

How many real alternatives exist, though? DuckDuckGo is based on Bing, afaik. So there are Google and Bing - any others?

There are also Baidu and Yandex, which dominate China and Russia, respectively. https://www.baidu.com https://yandex.com Qwant is a France-based search engine that has a similar privacy focus to DuckDuckGo. https://www.qwant.com Million Short tries to find sites that other search engines miss. https://millionshort.com Wikipedia has a more comprehensive list: https://en.wikipedia.org/wiki/List_of_search_engines

Thanks! Wikipedia still mentions Yahoo! - I thought they gave up and switched to Bing years ago?

Re: Building a Search Engine from Scratch

#119
post #90
post #11

Earlier quoted context omitted.

There is an autocorrect feature and it's even described in the article. Or you mean something else?

They mean auto complete.

Here is the quote from article "This not only involves some normalization, but also expansions and spell corrections, if necessary. "

Re: Building a Search Engine from Scratch

#120
post #46
post #29

That Cliqz is trying to actually build a new search stack is commendable. This is way more exciting to me than DuckDuckGo and other services that just package up Bing search results under different branding. I'm skeptical that they'll be successful, but I wish them the best. They should market (and engineer) strongly on privacy since that's where Google is weak.

« DuckDuckGo and other services that just package up Bing search results » Had to look that up. I found https://help.duckduckgo.com/results/sources/ Bing is just one of "hundreds of vertical sources delivering" results to DuckDuckGo.

So Duck Duck Go isn't actually a Search Engine, but an Interface or Proxy To Bing / or multiple search engine?

I have always thought DuckDuck were doing their own searches.

Post reply on HN