Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The world needs more search engines

0x65.dev

Post reply on HN
Print view View original

451–460 of 549 posts

Re: The world needs more search engines

#451

Post by kkm »

weinzierl wrote: post #422
pythux wrote: post #164

Earlier quoted context omitted.

Thanks for the feedback, if you find something fishy in the requests then that's like a bug and we will gladly fix it ASAP. Disclaimer: I work at Cliqz.

For the current Cliqz Browser (1.30.1) when I type "pythux" in the URL bar the request looks like this: GET /api/v2/results?nrh=1&q=pythuxs=UTlnJzAh4ULkORaiZgLPO6LW9LOWZZoO&n=5&qc=0&lang=en&locale=en-US&platform=0&o=%5B%5B%22custom-search%22%5D%5D&country=de&adult=0&loc_pref=ask&count=10&suggest=0 HTTP/1.1 Host: api.cliqz.com 1. I saw UTlnJzAh4ULkORaiZgLPO6LW9LOWZZoO in other requests too, but not all of them. Can yo…

Hi, Disclaimer: I work for Cliqz.

Thank you the questions, we are always looking for constructive feedback on and off HN.

1. These random values are used for grouping partial queries together, and they reset when you press enter or start a new query. Source code on how it's generated: https://github.com/cliqz-oss/browser-core/blob/master/module... We actually take one additional precaution of using crypto random and not plain Math.random(), which could potentially be used to link multiple sessions together.

2. There is no feature to disable auto-suggest. But I will pass your feedback to the team.

3. No there is no shared ownership, we don't have access to their servers. We also do additional encryption with bucketing on the payload sizes that we route through foxyProxy, so that the proxy provider cannot learn anything about the content of the message. We will have a blogpost explaining this on Wednesday - 4th December. Also, we are looking to add an option, where user can choose their own proxy provider too.

4. There are two parts: a. You can select the option from Control Center (Q menu) icon in the toolbar -> search -> search via proxy. (Now the calls should go through FoxyProxy) b. All calls to api.cliqz.com go through proxy when in private mode. The only reason it's not default is latency. As to what goes through FoxyProxy by default is: all Human Web data.

Once again, we appreciate you looking into details, and please keep digging, we would be happy to answer, improve our documentation and if there are bugs specially related to privacy and security they are on our uttermost priority.

Re: The world needs more search engines

#452

Post by jefftk »

close04 wrote: post #444
jefftk wrote: post #395

Earlier quoted context omitted.

Source: I work on ads at Google, and if we were doing this I'm pretty sure I'd know. If it was being used for targeting it would be practical to run an external study demonstrating that.

> external study demonstrating that I hope you understand the reticence of taking any statement coming from a Google employee defending Google with a grain of salt. You have defended Google in the past only to fall silent when presented with the actual study contradicting you. [0] It not that you aren't/can't be right, just that the "source" suggests some bias and Google's track record does not work in their favor. […

> You have defended Google in the past only to fall silent when presented with the actual study contradicting you.

That's not how I see that conversation:

* reaperducer was asking why it was useful for the browser to show that the page was one that usually loaded quickly/slowly

* As someone who had worked on an effort to speed up the web I replied with why I thought it was useful

* jfoster gave a good response describing why it might not have the effect I expected, since if users know a site is usually slow that may make them more patient

* I replied that this was still good, because users were in a position to make a better decision about whether to continue waiting for the site to load.

* You responded with something completely unrelated to what we were talking about.

* I tried to be helpful anyway, even though your comment wasn't something I knew much about.

* You continued in a direction that I don't know much about (how to communicate things like whether location tracking is on) and linked to a study which I didn't have time to read.

* This wasn't a discussion I was interested in, so I didn't respond. I don't see how the study you linked contradicted anything I was saying.

Re: The world needs more search engines

#453

Post by jefftk »

Nextgrid wrote: post #401
jefftk wrote: post #395

Earlier quoted context omitted.

Source: I work on ads at Google, and if we were doing this I'm pretty sure I'd know. If it was being used for targeting it would be practical to run an external study demonstrating that.

Apologies but I can’t really trust someone who works at a company whose best interests are to violate people’s privacy, confirmed by all the dark patterns (both on the web and in Android) and their lack of GDPR compliance. I would be very curious as to how you’d prove this is or isn’t happening with a reasonable degree of accuracy considering all the factors involved in ad targeting. Unless you’re willing to give us…

> considering all the factors involved in ad targeting

An external study to evaluate whether Google is using fingerprinting would be some work, but pretty doable. Targeted advertising is generally very blunt: if someone thinks you're especially interested in a valuable category they'll often pay a lot to advertise to you. So you could set something up where test browsers visit pages related to high-value categories (mattresses, asbestos cancer, credit cards, ...), clear client-side data, and then visit a site that loads ad scripts only from Google (to make sure you're not getting someone else's fingerprinting) and see whether the ads differ from a control group that never visited those pages.

Re: The world needs more search engines

#454

Post by tschakkaMarc »

garaetjjte wrote: post #450
tschakkaMarc wrote: post #431

Earlier quoted context omitted.

Hi, Marc from Cliqz. This one haunts us (in HN and also in my dreams). Let's first get one thing clear: It was a terrible blog post. Second: The 1% who had Cliqz installed were actually safer than the ones without. Why? If you use a (most) browsers every keystroke in the URL bar gets send to Google. This is how autosuggest works. This was and is the case with Firefox also. Cliqz in the functionally implemented in Fir…

Obviously using search engine sends search queries to it.. okay. But that experiment was sending most of browsing history, without explicit opt-in. Collecting browsing history from users of popular browser without asking them is just evil. (and it seems you still defend that decision)

Yes, I’m defending it, because again: We took drastic steps to never send anything private (like checking within the browser whether the URL is unique or different if logged in or out and then never sending it, not to mention that there of course was no identifier and we made record linkage impossible, so no click profile, and much more). If in doubt we drop and don’t send. And again – there were tons of (pen) tests and scrutiny to make sure no private data point ever leaves your browser. It is built with the mindset “if it reaches our server, we should technically not be able to identify any single person or any surf pattern or any private URL” – this was and is also tested by many (privacy) researchers before and after the experiment. And again, all this was and is open source. This is way more than any industry standard, and I simply don’t know of any company that works with data that has a higher standard. Be our guest to validate it yourself. And please read our blog post Tuesday: we will explain how this is done. But if you simply oppose this (and similar methods from people who really care about privacy), you basically accept the status-quo of the worst data collectors, because no one else then will ever emerge (because you do need this kind of data to build a search).

[EDIT]: Just to clarify and not have anyone create the wrong idea - I defend my earlier point. But your question is loaded. Here's why: We do not collect browser history, which by definition implies being able to piece visited urls back to a profile in our servers. That is impossible - to us, each single URL comes as a detached datapoint - devoid of any information that can be used to aggregate them back to a user profile.

Re: The world needs more search engines

#455

Post by jammygit »

jefftk wrote: post #258
wolco wrote: post #210

Earlier quoted context omitted.

google records: ip address, browser fingerprinting. Try visiting from tor

For spam / bot / fraud detection, yes. For ad targeting, no.

How is this information logged/preserved, and for how long? Just because it isn’t being used for ads doesn’t mean a person is comfortable filling up a database with their activity.

IMO, ads are probably the least worrisome way the data could be used. A boring but scary example is that aol search history leak (which is still searchable today):

https://searchids.com/

This person is identified by name for example: https://searchids.com/user/19431784-joann_whitman

Re: The world needs more search engines

#456

Post by scarface74 »

summerlight wrote: post #438
scarface74 wrote: post #389

Earlier quoted context omitted.

So Apple is a “monopoly” over what exactly?

The entire ecosystem running on iOS/OS X. This is exclusive on Apple produced hardware, which effectively forms a single relevant market as you cannot easily substitute this with others even if there's a small but significant and non-transitory increase in price. https://en.wikipedia.org/wiki/Small_but_significant_and_non-...

So is that how you define monopoly now? Does that make all of the console makers monopolies over their platforms? Does that mean that Apple also has a “monopoly on voice assistants that can be used with AirPods”?

Does it also have a monopoly on apps that can be run on the “HomePods”? Does every smart TV manufacturer with their own OS also have a “monopoly” on their ecosystem?

Let’s go further down the rabbit hole. Does Tesla have a “monopoly” on software that can run on its cars?

If non insignificant switching costs defines a “monopoly” every software as a service app would have a “monopoly”.

Unfortunately, neither the EU or the US has ever defined “monopoly” like HN posters...

Re: The world needs more search engines

#457

Post by biasedOpinion »

seektable wrote: post #443

We don't need more search engines like google, we need a new kind of engines that will supersede google/bing/etc and break the following axioms: * search engine is a cloud service, which is not controlled by user of that engine * search gives final results in milliseconds (why? because google cannot spend many seconds/minutes for you, that's why) * search creates information bubble (that user cannot control) because…

Disclaimer: I work at Google so I'm 100% biased. All opinions are my own. Edit: formatting

>search engine is a cloud service, which is not controlled by user of that engine

So is hacker news.

>search gives final results in milliseconds (why? because google cannot spend many seconds/minutes for you, that's why)

Do you want latency to be larger??

>open source, because user should trust the code

That would make it easier for SEOs to game the system. Or perhaps it would put everyone on a level playing field. I'm not sure.

>self-hosted (easily deployed in a click), not a cloud SaaS.

Do you mean users should keep their own index of the whole web by themselves?

>more useful than google because of accumulated data about you that are processed by computational knowledge engine

First I think you greatly overestimate how useful information about you can be. Second if that worked then it would make they problem you mentioned before (information bubble) worse...

Re: The world needs more search engines

#458

Post by tschakkaMarc »

techslave wrote: post #448
tschakkaMarc wrote: post #96

Earlier quoted context omitted.

Hi, Marc from Cliqz (and one of the authors of the linked article): We at Cliqz do try and others (like mojeek in the UK or Seznam in Czech) are trying as well. But yes, it has very high costs. We did a lot of (I find smart) technical shortcuts (and will explain how in the next days in our blog), but it is still an investment. The main hurdle is however distribution. As said in the article, Google spends 1/4 of their…

search is not close to being a natural monopoly. you are vastly underestimating how strong google’s execution has been. perhaps because you need to convince yourself that you could compete, “if only” ...

One doesn't exclude the other. I admire a lot of what Google has done and agree on a very strong execution. This still doesn't change the fact, that a search engine (1) gets better with every query-click pair and hence favors the market leader, (2) is a highly profitable business, which allows to buy distribution (this is the biggest market entry barrier), and (3) there are many small points that add up (crawler access is one minor but annoying issue: Web-Sites very often see all crawlers except Google and Bing as "bad scrapers"; so small companies need to invest a lot of time to convince them to get access and many will still never allow it what users then rate as bad quality) ... but, yes, Google is doing a lot of things right, they have very good people. Many of them my friends or ex-colleagues. This still doesn't change the fact of the original article: A 93% market share is not great.

Re: The world needs more search engines

#459

Post by chrisshroba »

pythux wrote: post #164
weinzierl wrote: post #153

Earlier quoted context omitted.

Oh, the requests contained much more than the keystrokes and I looked into the anonymization claims as well. I'd have to dig out my Wireshark traces to tell you exactly what was transmitted, but I'm pretty sure it was enough to id the user. And by the way Cliqz never claimed that the anonymization happens at the endpoint, from what I remember they were very proud of their proprietary anonymization technology in the b…

Thanks for the feedback, if you find something fishy in the requests then that's like a bug and we will gladly fix it ASAP. Disclaimer: I work at Cliqz.

IP Addresses are generally pretty sticky and make a great unique id for users. Even without uids being sent in the request, keying on ip address should eliminate a ton of anonymity.

Re: The world needs more search engines

#460

Post by KirinDave »

green1 wrote: post #290
KirinDave wrote: post #280

Earlier quoted context omitted.

The US has what we might call "better" libel and slander laws than some actors in the EU and with the UK being a popular example of the most ridiculously bad laws historically.

>> Aside from [...] libel

I didn't see your edit, sorry.
Post reply on HN