Ask HN: Can we create a new internet where search engines are irrelevant? • Hacker News Board

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#161

Post by davemp » Wed, Jun 26, 2019, 1:59 PM UTC

jasode wrote: post #142

davemp wrote: post #128

Earlier quoted context omitted.

The problem is that current search engines are indexing what is essentially a stack of random books thrown together by anonymous library goers. Before being able to guide readers to books, librarians have to the following non-trivial tasks over the entire collection: - identify the book's theme - measure the quality of the information - determine authenticity / malicious content - remember the position of the book in…

>I welcome any attempts to build a more organized internet. I don't think the communal book pile approach is scaling very well. Let me know if I misunderstand your comment but to me, this has already been tried. Yahoo's founders originally tried to "organize" the internet like a good librarian. Yahoo in 1994 was originally called, "Jerry and David's Guide to the World Wide Web" [0] with hierarchical directories to cu…

> I often see several suggestions that the alternative to Google is curated directories but I can't tell if people are unaware of the early internet's history and don't know that such an idea was already tried and how it ultimately failed.

¿Por qué no los dos?

1) The idea is that a more organized structure is easier for a librarian to index. Today, libraries still have librarians. The book pile just wouldn't take decades to build familiarity.

2) Times change. New technology exists, people use the internet differently, and there's more at stake. Just because an approach didn't work before doesn't mean that it won't work now.

There are real problems with an organizational approach, but I don't see why the idea isn't worth a revisit.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#162

Post by swalsh » Wed, Jun 26, 2019, 2:00 PM UTC

pjc50 wrote: post #77

ntnlabs wrote: post #72

How about this: Internet as a service. Instead of looking for answers You will "broadcast" Your needs. Like "I need a study about cancer". And You will receive a list of sources that answered Your question. maybe with some sort of decentralised rating and maybe Country and author. How about that?

Broadcasting your searches seems, if anything, even worse for privacy, and an invitation for just-in-time spam.

That's a workable issue. If a random and unique guid is asking for results, it would be hard to correlate users.

Of course there would defiantely be an issue with how you generate the guid (for example if it was generated by the users MAC + some predictable random number generator that might be reversible). So you would keep that in mind. But these seem like workable issues.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#163

Post by tablethnuser » Wed, Jun 26, 2019, 2:04 PM UTC

joaobeno wrote: post #155

tablethnuser wrote: post #153

One way to replace search is to return to curation by trusted parties. Rather than anyone putting a web page up and then a passive crawler finding it and telling everyone about it, (why should I trust any search engine crawler,) we could "load" our search engine with lists of websites. These lists are published and maintained by curators that we have explicitly chosen to trust. When we type into the search box it can…

Kind of like the Good Old Days™? Jokes aside, Search engines are for stuff we don't know... I get my news from some sites, this one for example, and I don't depend o Google for that... But when I want to know about the viability of zipping a UTF-8 encoded text to save space on my DB, there is no way to get may answer without a search engine...

If there's anything the modern internet has taught me it's that the Good Old Days were doing some things right! We don't know how to scale trust to internet-sized communities yet so a little tribalism may be warranted.

The way this solution solves the I Don't Know What I Don't Know Problem is by making you curate your own list of experts. For your example query, a colleague may have told you about a popular list that thousands of DBAs subscribe and contribute to. So when you search that query it has the sites to crawl and find the material

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#164

Post by davemp » Wed, Jun 26, 2019, 2:06 PM UTC

PeterisP wrote: post #145

davemp wrote: post #128

Earlier quoted context omitted.

The problem is that current search engines are indexing what is essentially a stack of random books thrown together by anonymous library goers. Before being able to guide readers to books, librarians have to the following non-trivial tasks over the entire collection: - identify the book's theme - measure the quality of the information - determine authenticity / malicious content - remember the position of the book in…

The current search engines are also indexing books maliciously inserted in the library in a way to maximize their exposure e.g. a million "different" pamphlets advertising Bob's Bible Auto Repair Service inserted in the Bible category. A "better library" can't be permissionless and unfiltered; Dewey Decimal System relies on the metadata being truthful, and the internet is anything but. You can't rely on information p…

Perhaps there exists a happy medium between: manual curation -- unfiltered

PageRank is kind of a pseudo manual curation. The manual effort is just farmed out to the greater internet and analyzed.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#165

Post by worldsayshi » Wed, Jun 26, 2019, 2:06 PM UTC

fifnir wrote: post #63

worldsayshi wrote: post #44

Earlier quoted context omitted.

We could supercharge word of mouth. I've been thinking about an alternative upvote model where content is ranked not primarily based on aggregate voting but by: - ranking content that users you have upvoted higher - ranking content that users with similar upvote behaviour higher While there is a risk of upvote bubbles, it should potentially make it easier for niche content to spread to interested people and make it p…

> ranking content that users with similar upvote behaviour higher That's how you make echochambers

All social media have echo chamber characteristics. You have to counteract it with transparency and opt-in/out.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#166

Post by TeMPOraL » Wed, Jun 26, 2019, 2:07 PM UTC

Fjolsvith wrote: post #105

TeMPOraL wrote: post #91

Earlier quoted context omitted.

> By and large, people don't seem to be willing to pay for content on the web. Hence, advertising became the dominant business model for content on the web. I don't think the causality is right here. People might not be willing to pay for content on the web because advertising enables competitors to offer content for free. If you removed that option, if people had no choice but to pay, it might just turn out that peo…

Huh. That sounds like a free market model. Isn't this what different newspapers like NYT and WSJ are moving towards? Why can't both models coexist?

Because one totally destroys the other.

Slave labour, selling poison or dumping waste into rivers are all superior business models too, but that doesn't mean they should exist in a civilized society.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#167

Post by worldsayshi » Wed, Jun 26, 2019, 2:08 PM UTC

eterps wrote: post #64

worldsayshi wrote: post #44

Earlier quoted context omitted.

We could supercharge word of mouth. I've been thinking about an alternative upvote model where content is ranked not primarily based on aggregate voting but by: - ranking content that users you have upvoted higher - ranking content that users with similar upvote behaviour higher While there is a risk of upvote bubbles, it should potentially make it easier for niche content to spread to interested people and make it p…

I've had similar ideas recently. Especially niche content (or shared research) would probably be notoriously hard (WRT false positives) for machine learning to decide whether it is relevant to you, people with similar interests know that much better. I was also wondering what would be good options to store votes/upvotes in a decentralized way.

> people with similar interests know that much better

Yeah, I wonder if there is a cheap way to test this. Actually! There could be! Like using favorite's here on hacker news. That could be mined and visualized in various ways. (Although a quick sample shows me that it's a rarely used feature)

> I was also wondering what would be good options to store votes/upvotes in a decentralized way.

Yeah there are a lot of interesting optimization challenges if you really want to utilize upvote graphs for ranking.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#168

Post by TeMPOraL » Wed, Jun 26, 2019, 2:09 PM UTC

BjoernKW wrote: post #111

TeMPOraL wrote: post #91

Earlier quoted context omitted.

> By and large, people don't seem to be willing to pay for content on the web. Hence, advertising became the dominant business model for content on the web. I don't think the causality is right here. People might not be willing to pay for content on the web because advertising enables competitors to offer content for free. If you removed that option, if people had no choice but to pay, it might just turn out that peo…

How would you achieve that? By outrightly outlawing advertising? There absolutely are paid options on the web. It's just that they don't seem to appeal to a sufficient number of buyers so advertising could become irrelevant.

> How would you achieve that? By outrightly outlawing advertising?

Yes.

> There absolutely are paid options on the web. It's just that they don't seem to appeal to a sufficient number of buyers so advertising could become irrelevant.

They aren't appealing in the presence of ad-subsidized free alternatives. Remove the latter, and they just might become appealing again.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#169

Post by gexla » Wed, Jun 26, 2019, 2:10 PM UTC

KirinDave wrote: post #158

gexla wrote: post #136

Good question. I'm going to run an experiment. First "go-to" for search will be my browser history. As long as the site I know I'm looking for is in my browser history, then I'll go there and use the search feature to find other items from that site. Bookmark all the advanced search pages I can find for sites I find myself searching regularly. Resist mindless searching for crap content which usually just takes up tim…

> but I can at least see who is pushing the link. I can determine if this person is someone I would trust for recommendations. This doesn't seem true at all to me. Twitter dramatically shapes and modifies timelines to promote whatever they want. They're even more aggressive on modifying the search experience. All of those constraints are invisible. It's dangerous to think you have more control or insight there.

Twitter has a social graph and communities much like Reddit. This adds more information. There are people posting information who I trust and even know IRL.

> All of those constraints are invisible. It's dangerous to think you have more control or insight there.

And yet you are commenting as if these results aren't invisible to you? The machinery behind Google search results aren't invisible? Are you trying to say that one invisible thing is more "X" than another invisible thing?

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#170

Post by [deleted] » Wed, Jun 26, 2019, 2:14 PM UTC

ScottFree wrote: post #116

ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

> or you find a novel way to make a neutral search engine Just because nobody's said it in this thread yet: blockchain? I never bought into the whole bitcoin buzz, but using a blockchain as an internet index could be interesting.

[deleted]