Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can we create a new internet where search engines are irrelevant?

news.ycombinator.com

Post reply on HN
Print view View original

381–390 of 395 posts

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#381

Post by iblaine »

stakhanov wrote: post #259
iblaine wrote: post #228

Yes, it was called Yahoo and it did a good job of cataloging the internet when hundreds of sites were added per week: https://web.archive.org/web/19961227005023/http://www2.yahoo... I'm old enough to remember sorting sites by new to see what new URLs were being created, and getting to that bottom of that list within a few minutes. Google and search was a natural response to solving that problem as the number of sites…

You don't have to go all the way back into Yahoo-era when it comes to manually curated directories: DMOZ was actively maintained until quite recently, but ultimately given up for what seems like good reasons.

This is true, and DMOZ was used heavily by Google's earlier search algorithms to rank sites within Google. Early moderators of DMOZ had god like powers to influence search results.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#382

Post by aslaan »

Nasrudith wrote: post #234
luxuryballs wrote: post #222

Earlier quoted context omitted.

The reverse is a problem too, Google filtering things out based on their political leanings in an attempt to shape public opinion.

I haven't seen any examples which were anything other than runaway persecution complexes of those who found their world view was less popular than they believed - which were greeted with exasperation by testifying engineers who had to explain how absurdly unscaleable it would be to do it manually.

https://gohmert.house.gov/news/documentsingle.aspx?DocumentI...

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#383

Post by jppope »

jtolmar wrote: post #323
adrianmonk wrote: post #257

I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…

Some thoughts on the problem, not intended as a complete proposal or argument: * Indexing is expensive. If there's a shared public index, that'd make it a lot easier for people to try new ranking algorithms. Maybe the index can be built into the way the new internet works, like DNS or routing, so the cost is shared. * How fast a ranking algorithm is depends on how the indexing is done. Is there some common set of fea…

this is a pretty killer idea

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#384

Post by jppope »

_nalply wrote: post #14

The deeper problem is advertising. It is sort of a prisoner's dilemma: all commercial entities have a shouting contest to attract customer attention. It's expensive for everybody. If we could kill advertisement permanently, we can have an internet as described in the question. This will almost be like an emergent feature of the internet.

Promotion is a need, and a very important need for ideas to spread. We all know that the concept of "if you build it they will come" doesn't work". Google's adaptation for this was to make advertising relevant... which is actually a considerable improvement over historical media models...

There's a saying in sales: "people hate to be sold, but they love to buy"... which is akin to what you are saying here. Advertising isn't the problem... the problem is that the reasons why people are promoting aren't novel enough... (rent seeking... which creates noise)

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#385

Post by Sohcahtoa82 »

rmsaksida wrote: post #33
azangru wrote: post #11

I can't imagine how this is possible. Imagine I have a string of words (a quote from a book or an article, a fragment of an error message, etc), and I want to find the full text where it appears (or pages discussing it). How would you do that without a search engine?

I think OP's idea is that search services would be built into the Internet, and not provided by a third party. That is, when a website is published or updated, it is somehow instantly indexed and made available for search as a feature of the platform on which it was published.

> built into the Internet

Uh...what? How do you define this?

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#386

Post by cruano »

penagwin wrote: post #311
Retra wrote: post #278

Earlier quoted context omitted.

Couldn't you just randomize result ordering?

You know how google search results can get really useless just a few pages in? And it says it found something crazy like 880,000 results? Imagine randomizing that. --- Unrelated I searched for "Penguin exhibits in Michigan". Of which we have several. It reports 880,000 results but I can only go to page 12 (after telling it to show omitted results). Interesting... https://www.google.com/search?q=penguin+exhibits+in+mi…

I heard that they also filter results by some undisclosed parameters, like they don't show you anything that hasn't been modified in the last ~10 years, no matter how hard you try

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#387

Post by sogen »

allworknoplay wrote: post #284
adrianmonk wrote: post #257

I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…

Spitballing here, but what if instead of a monolithic page rank algorithm, you could combine individually maintained, open set rankings? ===Edit=== I mean to say you as the user would gain control over the ranking sources, the company operating this search service would perform the aggregation and effectively operate marketplace of ranking providers. ===end edit=== For example, one could be an index of "canonical" si…

Sounds like ad-Blocker repos, nice!

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#388

Post by jgtrosh »

ergothus wrote: post #359
jtolmar wrote: post #323

Earlier quoted context omitted.

Some thoughts on the problem, not intended as a complete proposal or argument: * Indexing is expensive. If there's a shared public index, that'd make it a lot easier for people to try new ranking algorithms. Maybe the index can be built into the way the new internet works, like DNS or routing, so the cost is shared. * How fast a ranking algorithm is depends on how the indexing is done. Is there some common set of fea…

> Indexing is expensive. If there's a shared public index, that'd make it a lot easier for people to try new ranking algorithms. Maybe the index can be built into the way the new internet works, like DNS or routing, so the cost is shared. But what are you storing in your index? The content that is considered in your ranking will vary wildly by your ranking methods. (example - early indexes cared only for the presence…

> The only way to store an index of all content (to cover all the options) is to...store the internet.

An index cannot hope to cover all options, these ideas are antithetical.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#390

Post by smitop »

mavsman wrote: post #287
adrianmonk wrote: post #257

I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…

How about open sourcing the ranking and then allowing people to customize it. I should be able to rank my own search results how I want to without much technical knowledge. I want to rank my results by what is most popular to my friends (Facebook or otherwise) so I just look for a search engine extension that allows me to do that. This could get complex but can also be simple if novices just use the most popular rank…

If the details of a ranking algorithm are open source, it would be easy to manipulate them.
Post reply on HN