google's suggested autocorrect is one it's most impressive features; idk I'd say the relevance of the search results almost comes in a near second to that. so make a competitive "suggested autocorrect" solution and then I think you'd have a stew going.
Building a Search Engine from Scratch
11–20 of 151 posts
Re: Building a Search Engine from Scratch
#12Re: Building a Search Engine from Scratch
#13I don’t see how this will ultimately be successful. They’re basically reverse engineering Google by looking at user logs. Google will always have a leg up here because they have all the Google data. And even if it does work for a while, there still needs to be the original signal to copy. Someone will have to crawl the web and index content. I’m super eager to find new approaches to search, but another Google clone i…
A sufficiently strict intellectual-property regime might find this a copyright violation. But without any inside info, I strongly suspect Google & other incumbents already do similarly-indirect modeling of their competitors' behavior, via extensive query/click-trail mining, in ways that ultimately feed into improvements of their own systems. So, they might not want to press the issue.
Still, this creates a dependency on the competitor you were hoping to displace, where most of your earlier values comes from "drafting" in the easy-path they've already cleared.
Re: Building a Search Engine from Scratch
#14https://anolysis.privacy.cliqz.com/
As you can see in this subthread, they claim it's "anonymized" analytics,
https://news.ycombinator.com/item?id=21718694
which, as 99% of research suggests, isn't anonymous at all:
https://www.fastcompany.com/90278465/sorry-your-data-can-sti...
Re: Building a Search Engine from Scratch
#15Cliqz still hasn't given a good answer as to why they use what's seemingly a typo in 'analysis' to get around user tracker blockers. https://anolysis.privacy.cliqz.com/ As you can see in this subthread, they claim it's "anonymized" analytics, https://news.ycombinator.com/item?id=21718694 which, as 99% of research suggests, isn't anonymous at all: https://www.fastcompany.com/90278465/sorry-your-data-can-sti...
Hi, I read the thread and thought the answer was good enough, but it seems that you are not yet convinced. Let me try:
1) Here there is a list of publications regarding privacy by Cliqz (including published scientific papers). It should have fairly easy to find it using a search engine :-) https://0x65.dev/pages/dissemination-cliqz.html Hopefully, the paper will convince you that Cliqz privacy commitment is serious.
2) Feel free to monitor your own traffic to see whether or not we are tracking you.
3) Honestly, if someone tells you that anolysis means anonymous + analysis, why do you not believe it? It does not take long to find references of the name on the source code. On a separate note, as a company (Cliqz) that offers anti-tracking and ad-blocking, I can tell you that blocklists are a bit more sophisticated than that.
Hope that this will address your concerns,
[comment edited: why do you not believe it?]
Re: Building a Search Engine from Scratch
#16Can someone comment on how to use knowledge graphs for search? I have seen some applications in NLP but I am curious how it can tie in with traditional search.
Re: Building a Search Engine from Scratch
#17I don’t see how this will ultimately be successful. They’re basically reverse engineering Google by looking at user logs. Google will always have a leg up here because they have all the Google data. And even if it does work for a while, there still needs to be the original signal to copy. Someone will have to crawl the web and index content. I’m super eager to find new approaches to search, but another Google clone i…
Re: Building a Search Engine from Scratch
#18I don’t see how this will ultimately be successful. They’re basically reverse engineering Google by looking at user logs. Google will always have a leg up here because they have all the Google data. And even if it does work for a while, there still needs to be the original signal to copy. Someone will have to crawl the web and index content. I’m super eager to find new approaches to search, but another Google clone i…
Indeed, using the (query -> result) mapping in their 'Human Web' logs, from other search engines, is a roundabout way of copying both the simple word-to-doc-index, and the complicated ranking logic, of other search engines. A sufficiently strict intellectual-property regime might find this a copyright violation. But without any inside info, I strongly suspect Google & other incumbents already do similarly-indirect mo…
Copying, learning would be a bit more precise, and I'm not kidding. We do not answer queries 1 to 1, query-logs are used to build a more concise model of the page. It's not just a cache, we started like this, but we quickly learn to answer unseen queries.
A sufficient strict IP might find text snipped a copyright violation too. It's a trick area. Personally, I'm at peace as we get the content of web pages, as everyone else.
As for the dependency, it would be if we were not able to generate our own synthetic queries, which we are. So even if all other search engines of the world were to disappear, we would still be able to operate. That was not always the case, as you pointed out.
Re: Building a Search Engine from Scratch
#19I don’t see how this will ultimately be successful. They’re basically reverse engineering Google by looking at user logs. Google will always have a leg up here because they have all the Google data. And even if it does work for a while, there still needs to be the original signal to copy. Someone will have to crawl the web and index content. I’m super eager to find new approaches to search, but another Google clone i…
The article describes techniques used by all search engines, not just Google. Search engines have existed before Google, and despite Google's monopoly on search, "Google clone" is a poor term to describe all search engines when alternatives with unique features (e.g. DuckDuckGo) exist.
Re: Building a Search Engine from Scratch
#20Can someone comment on how to use knowledge graphs for search? I have seen some applications in NLP but I am curious how it can tie in with traditional search.