Live data from Hacker News

A New Search Engine

0x65.dev

91–100 of 107 posts

Re: A New Search Engine

#91
post #38

If you are going to make a new search engine, you need to attack a problem that people have, like Duckduckgo solving privacy issues. I don't want to install something that collects a bunch of personal info about me. A better idea is to search bookmarked sites and the cache. And do it locally.

To be fair, Mojeek addressed the search engine privacy problem long before DDG existed.

Re: A New Search Engine

#92
post #11
post #6

They talk about using query logs to optimize their search results: >Queries performed by people, if associated to a web page, serve as even cleaner summaries than anchor text. This is because all the logic put in place by the search engine, who resolved the query with a list of web pages, and all human understanding and experience that led one to select the best page from the offered result list end up embedded in th…

Surfacing new content in search engines is a very challenging problem. I am guessing they use a combination of social signals (twitter, facebook) popularity and domain popularity amongst other signals.

[Disclaimer: I work at Cliqz]

You are on point. Recency is a challenging problem in multiple ways for search engines. Not just limited to discovering new content, but also how does one index it? How does one balance out when you have for the same query "very new", "new", "slightly old" and "really old" results during ranking. This involves both news as well as new webpages surfacing on the web.

On top of this, we have to remember that this is a fully autonomous real time system which requires solving some of the most difficult engineering challenges at scale and at the same time being mindful of the latency and quality constraints.

At the end of the day, it's all about the final user experience that we ship. We are very much mindful of the same. We will be publishing more details about Cliqz search, on our blog https://0x65.dev/ in the coming days, so stay tuned.

Re: A New Search Engine

#93
post #23

Earlier quoted context omitted.

I get the following error: This site can’t be reached0x65.dev refused to connect. Try: Checking the connection Checking the proxy and the firewall ERR_CONNECTION_REFUSED This happens on Chrome, Firefox, Safari, and Opera on my Mac.

First, let's check if you can open another domain on .dev TLD, like web.dev, if not then: Seems like you have some mapping for .dev TLD. Assuming based on your mention of Safari, that you are using Mac. Could you check if you have some setting in your /etc/resolver for dev TLD, or if you are using some service like dnsmasq which is trying to resolve .dev to a non-existent location.

Oh that's weird, I have "nameserver 127.0.0.1" under /etc/resolver/dev

I am on mac but I didn't touch anything. Is this how mac ships by default? Or do you think some app may have created this file?

Re: A New Search Engine

#94
post #88

Earlier quoted context omitted.

I'm afraid this will remove any results from page :-D

Is it really that bad? I surely hope it's not. I noticed that even Wikipedia is reported as having some trackers. But when I looked closer I noticed that most of those belong to the Wikimedia foundation, which is fine. I mean, I don't mind site owners tracking what I do on their site, I just don't want to be followed across the whole Web. The rest of the Wikipedia trackers are supposed to be Google fonts and statics,…

Hi, I work at Cliqz on our Anti-tracking system, and the WhoTracks.Me data that powers these stats on the search page.

These stats are updated monthly, and based on millions of loads of each site. The WhoTracks.Me page for wikipedia.org (https://whotracks.me/websites/wikipedia.org.html) shows that the Google Fonts and Google Static trackers occur very infrequently (While the Wikimedia tracker may seem innocuous, they do set a cookie that is sent in third-party contexts, and have presence across several sites beyond Wikipedia (133 of the top 10k) (https://whotracks.me/trackers/wikimedia.org.html). Theoretically, they could track user sessions across these sites. In reality this is likely an oversight in the server configuration, but objectively this profile looks no different to that of a legitimate tracker.

Re: A New Search Engine

#96

Earlier quoted context omitted.

Is it really that bad? I surely hope it's not. I noticed that even Wikipedia is reported as having some trackers. But when I looked closer I noticed that most of those belong to the Wikimedia foundation, which is fine. I mean, I don't mind site owners tracking what I do on their site, I just don't want to be followed across the whole Web. The rest of the Wikipedia trackers are supposed to be Google fonts and statics,…

Hi, I work at Cliqz on our Anti-tracking system, and the WhoTracks.Me data that powers these stats on the search page. These stats are updated monthly, and based on millions of loads of each site. The WhoTracks.Me page for wikipedia.org ( https://whotracks.me/websites/wikipedia.org.html ) shows that the Google Fonts and Google Static trackers occur very infrequently ( While the Wikimedia tracker may seem innocuous, t…

Thanks for the explanation. It makes sense now. And that's ... depressing.

Re: A New Search Engine

#97

The nationalism on the homepage is a little odd... in particular since they're still essentially building on top of Google. https://cliqz.com/en/ > Europe has failed to build its own digital infrastructure. US companies such as Google have thus been able to secure supremacy. They ruthlessly exploit our data. They skim off all profits. They impose their rules on us. In order not to become completely dependent and end…

There's been a lot of discussion about how US-centric the internet is in general even discounting how many massively popular internet companies are US based. I don't think it's unreasonable for Europeans or other nations to try to be less dependent on the US and US based services. As an American, I think it's the smartest thing they can do and I welcome it.

As far as that goes, I agree.

I have an uncomfortable feeling, however that this is when the walls really start going up in the internet, beyond just the dictatorships.

Re: A New Search Engine

#98
post #73
post #25

Earlier quoted context omitted.

[Disclaimer, I work at Cliqz] I cannot answer for the "true" motivation of the investors, but their pitch and actions so far are well align with the fight against monopolies narrative. Do they want to get return on investment (eventually)? I would assume so, and I believe it would be fair. I do not see them as mutually exclusive. Of course, this is my personal opinion.

It’s a good investment as a hedge in the case the EU regulators kick Google in the ass hard enough.

The funny thing is that QWant is currently challenged on its ability to monetize its search engine. Short answer (for the moment): it can't. https://www.lemonde.fr/economie/article/2019/12/04/le-google... (in french)

Re: A New Search Engine

#99
post #26

> The experts, who chose to answer, suggested that we should first start with crawling the whole web. We were told that this would take between 1 and 2 years to complete, and would cost a minimum of $1 billion Why are costs so high for crawling?

Wouldn’t common crawl content be enough? If not what are the issues?

No, it's not enough and had poor coverage outside of USA. We have also answered this question (it appears to be popular) in today's post about technical details of our search https://0x65.dev/blog/2019-12-06/building-a-search-engine-fr...

Re: A New Search Engine

#100
post #95

Weird, your domain (cliqz.com) was blocked by my pihole.

Yes, we are on some minor blocking lists because of our data collection, even though is anonymous (please check the articles about Human Web on https://0x65.dev/) sending data, no matter what data, is a sin that has to be punished. A disservice to you ask me, but what can we do. [Disclaimer: I work at Cliqz]
Post reply on HN