Live data from Hacker News

Ask HN: Can we create a new internet where search engines are irrelevant?

news.ycombinator.com

131–140 of 395 posts

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#131
post #117

Earlier quoted context omitted.

> or you find a novel way to make a neutral search engine Just because nobody's said it in this thread yet: blockchain? I never bought into the whole bitcoin buzz, but using a blockchain as an internet index could be interesting.

even better, have something like git for the web - effectively working as an archive.

The problem with git is countering nefarious forces. The blockchain is better in that regard because the consensus algorithm can be used to verify that the listings are legitimate.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#132
post #122

I've had this idea floating in my head for a while, that one thing that might make the world better is some kind of distributed database, and a gravitation back to open protocols (though instead of RFC's... maybe we could maintain an open source library for the important bits) I was thinking the architecture of DNS is a good starting point. From there we can create public indexes of data. This includes searchable dat…

You've just more or less described Solid. https://solid.mit.edu/

I think a lot of people dismiss Solid based on its deep origins in Semantic Web, or because it's a slow project, based on Web standards, intended to solve long term problems.

But being part of the Web is a huge process, and with DIDs it maps just fine into decentralized worlds.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#134
tl;dr the problems are 1) relevancy, 2) integrity, 3) content management/curation.

If you've ever tried to maintain a large corpus of documentation, you realize how incredibly difficult it is to find "information". Even if I know exactly what I want.... where is it? With a directory, if I've "been to" the content before, I can usually remember the path back there... assuming nothing has changed. (The Web changes all the time) Then if you have new content... where does it go in the index? What if it relates to multiple categories of content? An appendix by keyword would get big, fast. And with regular change, indexes become stale quickly.

OTOH, a search engine is often used for documentation. You index it regularly so it's up to date, and to search you put in your terms and it brings up pages. Problem is, it usually works poorly because it's a simple search engine without advanced heuristics or PageRank-like algorithms. So it's often a difficult slog to find documentation (in a large corups), because managing information is hard.

But if what you actually want is just a way to look up domains, you still need to either curate an index, or provide an "app store" of domains (basically a search engine for domain names and network services). You'd still need some curation to weed out spammers/phishers/porn, and it would be difficult to find the "most relevant" result without a PageRank-style ordering based on most linked-to hosts.

What we have today is probably the best technical solution. I think the problem is how it's funded, and who controls it.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#135
post #117

Earlier quoted context omitted.

even better, have something like git for the web - effectively working as an archive.

The problem with git is countering nefarious forces. The blockchain is better in that regard because the consensus algorithm can be used to verify that the listings are legitimate.

content change signed by creators private key, otherwise merge is rejected?

or, wiki approach...

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#136
Good question. I'm going to run an experiment.

First "go-to" for search will be my browser history.

As long as the site I know I'm looking for is in my browser history, then I'll go there and use the search feature to find other items from that site.

Bookmark all the advanced search pages I can find for sites I find myself searching regularly.

Resist mindless searching for crap content which usually just takes up time as my brain is decompressing from other tasks.

For search which is more valuable to me, try starting my search from communities such as Reddit, Twitter or following links from other points in my history.

Maybe if it's not worth going through the above steps, then it's not valuable enough to look up?

NOTE: Sites such as Twitter may not be much better than Google, but I can at least see who is pushing the link. I can determine if this person is someone I would trust for recommendations.

I bet if I did all of the above, I could put a massive dent in the number of search engine queries I do.

Any other suggestions?

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#137
post #120
post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

A few years ago there was that blogs thing, with rss... all things that favoured federation, independent content generation, etc. Now it's all about platforms. I understand that "regular people" are more comfortable with Facebook but, other than that, why are blogs and forums less popular now?

The problem with forums is that you end visiting 5~10 different forums, each with their own login, and some of them might be restricted at work (not that you should visit them often).

So it's easier to have 2~4 aggregators in where all the information you desire resides, even if in each of them there are different forums.

A unified entry point helps adoption.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#138

That was what the early internet was like (I was there). People built indexes by hand, lists of pages on certain topics. There was the Gopher protocol that was supposed to help with finding things. But this was all top-down stuff, the first indexing/crawling search engines were bottom-up and it worked so much better. And for a while we had an ecosystem of different search engines until Google came along, was genuinel…

Search is the problem. If you don’t rank in google you don’t exist on the internet. There is an entire economy built on manipulating search that is pay to play in addition to google continually focusing on paid search of natural SERPs. Controlling search right now is controlling the internet.

Whatever you replace Search with would be gamed in the same way.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#139
post #39

Earlier quoted context omitted.

But you still need a third party to rank the results. I don't just want any page about my error message, I want the best page.

The page rank could be a transparent algorithm, which is regularly updated by a consortium like W3C. The question is whether this would work in an adversarial setting where every party tries to inflate their page rankings by any trick they can find.

Not a chance it would survive. Google has enough problems fighting SEO right now and they don't publish their algorithm and have incredibly deep pockets.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#140
post #122

I've had this idea floating in my head for a while, that one thing that might make the world better is some kind of distributed database, and a gravitation back to open protocols (though instead of RFC's... maybe we could maintain an open source library for the important bits) I was thinking the architecture of DNS is a good starting point. From there we can create public indexes of data. This includes searchable dat…

Unless there's a significant change in human behaviour, convenience is always going to trump everything else including privacy - we have seen over and over again that people will happily hand over their personal data in return for a free service that is simple to use. So any solution where you control your own data is going to have to be as convenient as alternatives, otherwise there'll be an opening for a new centralised "we'll do all the hard work of owning your data for you" mega corp tech titan.
Post reply on HN