Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can we create a new internet where search engines are irrelevant?

news.ycombinator.com

Post reply on HN
Print view View original

221–230 of 395 posts

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#221

Post by izendejas »

basch wrote: post #127
ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

They are also a spam filter. It's not just an index of whats relevant, but removal of what maliciously appears to be relevant at first glance.

This. Everyone's missing the point of a search engine.

We're talking about billions of pages and if not ranked (authority is a good hueristic), filtered (de-ranked), etc then good luck finding valuable information because everyone is gaming the systems to improve their ranking.

I think this is part of the reason you get a lot of fake news on social media. It's a constant stream of information (a new dimension of time has been added to the ranking, basically) that needs to be ranked and with humans in the loop, there's no way to do this very easily without filtering for noise and outright malicious content.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#222

Post by luxuryballs »

izendejas wrote: post #221
basch wrote: post #127

Earlier quoted context omitted.

They are also a spam filter. It's not just an index of whats relevant, but removal of what maliciously appears to be relevant at first glance.

This. Everyone's missing the point of a search engine. We're talking about billions of pages and if not ranked (authority is a good hueristic), filtered (de-ranked), etc then good luck finding valuable information because everyone is gaming the systems to improve their ranking. I think this is part of the reason you get a lot of fake news on social media. It's a constant stream of information (a new dimension of time…

The reverse is a problem too, Google filtering things out based on their political leanings in an attempt to shape public opinion.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#223

Post by gerash »

anchpop wrote: post #211
thekyle wrote: post #195

Earlier quoted context omitted.

What you are describing is exactly what Google Contributor is trying to do. We'll have to see how it turns out. https://contributor.google.com/v/beta

It says it only works with "participating sites". I wonder why

The first version worked exactly as you proposed. The UX however was meh. You'd place a monthly limit on your ad (outbidding) spend (eg. $2) and it ended up outbidding only some of the ads: those served by Google which were also outbid by your amount.

So from a user's perspective it didn't fully work. Also the ad space wasn't fully removed (perhaps due to technical reasons) but was replaced with a blank image. It also didn't catch on much.

So they tried to pivot and now the program works with certain cooperating websites to fully get rid of all ads but I'm sure bigger websites would rather be in total control of monetizing themselves and can spend on the necessary IT infra. similar to most online newspapers these days.

I think an advertiser (eg. a legal firm) might be willing to pay eg. $10 per ad impression but no user is willing to outbid it so I think the first model (outbid in the auction) is more sustainable and profitable for both parties but needs to have all ad exchanges on board.

So in short, it's been tried but wasn't an instant (or even a slow) success and idk whether Google will continue investing in it or not.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#224

Post by vbsteven »

I was recently thinking about an open search protocol with some federation elements in two parts. A frontend and an indexer. The idea is that anyone can run his own search frontend or use a community hosted one (like Matrix). And then each frontend has X amount of indexers configured.

Each indexer is responsible for a small part of the web and by adding indexers you can increase your personal search area. And there is some web of trust going on.

Entities like stackoverflow and Wikipedia and reddit could host their own domain specific indexers. Others could be crowdsourced with browser extensions or custom crawlers and maybe some people want to have their own indexer that they curate and want to share with the world.

It will never cover the utility and breadth of Google Search but with enough adoption this could be a nice first search engine. With DDG inspired bang commands in the frontend you could easily retry a search on Google.

With another set of colon commands you can limit a search to one specific indexer.

The big part I am unsure about in this setup is how a frontend would choose which indexers to use for a specific query. Obviously sending each query to each indexer will not scale very well.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#225

Post by hayksaakian »

If you look at usage patterns, social media has replaced search engines for many use cases.

For example, if you want to know where to eat tonight, instead of searching "restaurants near me" you might ask your friends "where should I eat tonight" and get personalized suggestions.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#226

Post by zaphar »

davemp wrote: post #128
ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

The problem is that current search engines are indexing what is essentially a stack of random books thrown together by anonymous library goers. Before being able to guide readers to books, librarians have to the following non-trivial tasks over the entire collection: - identify the book's theme - measure the quality of the information - determine authenticity / malicious content - remember the position of the book in…

The really hard part of this to scale is the quality metric. Google was the first to really scale quality measurement by outsourcing it to the web content creators themselves.

Any attempt to create a decentralized index will need to tackle the quality metric problem.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#228

Post by iblaine »

Yes, it was called Yahoo and it did a good job of cataloging the internet when hundreds of sites were added per week: https://web.archive.org/web/19961227005023/http://www2.yahoo...

I'm old enough to remember sorting sites by new to see what new URLs were being created, and getting to that bottom of that list within a few minutes. Google and search was a natural response to solving that problem as the number of sites added to the internet grew exponentially...meaning we need search.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#229

Post by Nasrudith »

ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

The only assured way to have a "neutral" search engine is to run your own spiders and indexers which you understand completely.

Neutral in that sense is only "not serving the agenda or judgement of another" at the obvious cost of labor and not just as a one off thing as the searched content often attempts to optimize for views. It isn't like a library of passive books to sort through but a Harry Potter wizard portrait gallery full of jealous media vying for attention.

And pendantically it isn't true neutral - but serves your agenda to the best of your ability. A "true neutral" would serve all to the best of their ability.

Besides neutrality in a search engine on a literal level is oxymoronic and self defeating - its whole function is to prioritize content in the first place.

Post reply on HN