There's been a few mentions of the PageRank algorithm already...FWIW, Google's patent just expired. https://patents.google.com/patent/US6285999B1/en
Ask HN: Can we create a new internet where search engines are irrelevant?
351–360 of 395 posts
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#352I think Apple's current approach, where all the smarts (Machine Learning, Differential Privacy, Secure Enclave, etc.) reside on your device, not in the cloud, is the most promising. As imagined in so much sci-fi (eg. the Hosaka in Neuromancer) you build a relationship with your device which gets to know you, your habits and, most importantly in regard to search, what you mean when you search for something and what re…
Or even an online search engine that was configurable where you could customize the search engine and assign custom weights to different aspects.
I'd love to be able to configure rules like:
+2 weight for clean HTML sites with minimal Javascript
+5 weight for .edu sites
-10 weight for documents longer than 2 pages
-5 weight for wordy documents
I'd also like to increase the weight for hits on a list of known high quality sites. Either a list I maintain myself, or one from an independent 3rd party.
Once upon a time I tried to use Google's custom search engine builder with only hand curated high quality sites as my main search engine. It was to much trouble to be practical, but I think that could change with an actual tool.
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#353Yes, we need search engines, but they don't need to be monolithic. Imagine that indexing the text of your average web page takes up 10k. Then you get 100.000 pages per Gig. It means that you if you spend ~270USD on a consumer 10 tera drive you can index a billion webpages. Google no longer says how many pages they index, but its estimated to be with in one order of magnitude of that. This means that in terms of hardw…
How would you begin indexing everything?
This is good question. Crawling and storing the pages is the easy part... searching them with a sub 1 second response time is much harder. Which current DB platforms can handle the size of data that Google indexes?
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#354Can you walk through a complete use case?
A user wants to find a "relevant document".
What is that? What information does the user provide to specify the document?
Why does the user trust the result?
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#355I still remember gopher https://en.wikipedia.org/wiki/Gopher_(protocol)
Some of us still use it...
Shameless plug: http://www.jaruzel.com/gopher/gopher-client-browser-for-wind...
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#356I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…
This is a bang on distillation of the problem (or at least one way to view the problem, per "who controls the ranking and how it's defined").
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#357Earlier quoted context omitted.
Whatever you replace Search with would be gamed in the same way.
true, but when it was lycos, hotbot, altavista, google, webcrawler, aol, gopher, archy, usenet and so many other sources it was much easier to exist in many ways (harder to dominate) - people used to ‘surf the web’, join “webrings” and share stuff.. now they consume and post memes. so i blame behavior as much as monopoly
A lot of other things have changed since then, so the difference in tone you are noticing might not have much to do with search engines. In 1996 there were only about 16 million people on the internet, and usage obviously skewed towards the more technical nerdy crowd. Now there are 4,383 million people on the internet. Which is about 57% of everyone.
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#358Earlier quoted context omitted.
It should, but if anyone knows who these kingmakers are, it's still probably just a matter of time before they accrue enough power for it to be worth someone's time to at least try to track them down and manipulate their decisions (bribe, blackmail, sponsor, send free trials, target with marketing/propaganda campaigns, etc.)
Who says it even has the same kingmakers every day? Slashdot solved that part of metamoderation two decades ago. A person might be an expert in cars but not horses. A car expert might be superseded . The seed data creators could be a fluid thing.
This is a technocracy. Noone wants this but Hacker News.
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#359I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…
Some thoughts on the problem, not intended as a complete proposal or argument: * Indexing is expensive. If there's a shared public index, that'd make it a lot easier for people to try new ranking algorithms. Maybe the index can be built into the way the new internet works, like DNS or routing, so the cost is shared. * How fast a ranking algorithm is depends on how the indexing is done. Is there some common set of fea…
> Indexing is expensive. If there's a shared public index, that'd make it a lot easier for people to try new ranking algorithms. Maybe the index can be built into the way the new internet works, like DNS or routing, so the cost is shared.
But what are you storing in your index? The content that is considered in your ranking will vary wildly by your ranking methods. (example - early indexes cared only for the presence of words. Then we started to care about the count of words, then the relationships between words and the context. Then about figuring out if the site was scammy, or slow.
The only way to store an index of all content (to cover all the options) is to...store the internet.
I'm not trying to be negative - I feel very poorly served by the rankings that are out there, as I feel on 99% of issues I'm on the longtail rather than what they target. But I can't see how a "shared index" would be practical for all the kinds of ranking algorithms both present and future.
Re: Ask HN: Can we create a new internet where search engines are irrelevant?
#360I see a lot of good comments here, I got inspired to write this: What if this new Internet instead of using URI based on ownership (domains that belong to someone), would rely on topic? In examples: netv2://speakers/reviews/BW netv2://news/anti-trump netv2://news/pro-trump netv2://computer/engineering/react/i-like-it netv2://computer/engineering/electron/i-dont-like-it A publisher of webpage (same html/http) would pu…
Another inspiration: DNS for searching. What if we implement DNS-like protocol for searching. Think of recursive DNS. Do you have "articles about pistachio coloured usb-c chargers"? Home router says nope, ISP says nope, Cloudflare says nope, let's scan A to Z. Eventually someone gives an answer. This of course can (must?) be cached, just like DNS. And just like DNS, it can be influenced by your not-so-neutral browser…
The proliferation of Black hat SEOs would render this useless.