Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can we create a new internet where search engines are irrelevant?

news.ycombinator.com

Post reply on HN
Print view View original

241–250 of 395 posts

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#241

Post by roveo »

gexla wrote: post #136

Good question. I'm going to run an experiment. First "go-to" for search will be my browser history. As long as the site I know I'm looking for is in my browser history, then I'll go there and use the search feature to find other items from that site. Bookmark all the advanced search pages I can find for sites I find myself searching regularly. Resist mindless searching for crap content which usually just takes up tim…

I think you're onto something.

You could create a local search index built around your browser history. Then you could create a digital fingerprint-profile around it (still local). And then query other people's histories, that are similar to yours, in a DHT-address fashion.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#242

Post by msla »

davemp wrote: post #185
toast0 wrote: post #178

Earlier quoted context omitted.

There are plenty of these, wikipedia has a list [1]. I think these efforts get bogged down in the huge amount of content out there, the impermanence of that content and also the difficulty in placing sites into ontologies. And at the end of the day, there's not a large enough value proposition to balance the immense effort. I think, if you were to do it today, you would want to work on / with the internet archive, so…

Obviously a naïve web directory isn't going to cut it. What would make the approach viable is if there were a nice way to automate and crowd source most/all of the effort. Maybe that means changing the idea of what makes a website. Maybe there could just be little grass roots reddit-esque communities that are indexed/verified (google already favors reddit/hn links). Who knows, but it's an interesting problem to kick…

> Maybe there could just be little grass roots reddit-esque communities that are indexed/verified

Verified by who, exactly?

I know, I know... "dismissive comment", but it's an important thing to think about: Who decides what goes in the library? It's an evergreen topic, even in real, physical libraries, as those tedious lists of "Banned And Challenged Books" attest. It seems every time a copy of Huckleberry Finn gets pulled from an elementary school library in Altoona everyone gets all upset, so can you imagine what would happen if the radfems got their hands on a big Web Directory and cleansed it of all positive mentions of trans people?

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#243

Post by lefstathiou »

My approach to answering this would entail:

1) Determining what percentage of search engine use is driven by the need for a short cut to information you know exists but dont feel like accessing the hard way

2) Information you are actually seeking.

My initial reaction is that making search engines irrelevant is a stretch. Here is why:

Regarding #1, the vast majority of my search activity involves information I know how and where to find but seek the path of least resistance to access. I can type in "the smith, flat iron nyc" and know I will get the hours, cross street and phone number for the Smith restaurant. Why would I not do this instead of visiting the yelp website, searching for the Smith, set my location in NYC, filtering results etc. Maybe I am not being open minded enough but I don't see how this can be replaced short of reading my mind and injecting that information into it. There needs to be a system to type a request and retrieve the result you're looking for. Another example, when I am looking for someone on LinkedIn, I always google the person instead of utilizing LinkedIn's god awful search. Never fails me.

2. In the minority of cases I am looking for something, I have found that Google's results have gotten worse and worse over the years. It will still be my primary port of call and I think this is the workflow that has potential disruption. Other than an Index, I dont know what better alternatives you could offer.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#244

Post by greglindahl »

ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

The library index only indexes the information that fits on a card catalog card. That's extremely unlike a web search engine.

If you'd like to see an experimental discovery interface for a library that goes deeper into book contents, check out https://books.archivelab.org/dateviz/ -- sorry, not very mobile friendly.

Not surprisingly, this book thingie is a big centralized service, like a web search engine.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#245

Post by greglindahl »

tablethnuser wrote: post #153

One way to replace search is to return to curation by trusted parties. Rather than anyone putting a web page up and then a passive crawler finding it and telling everyone about it, (why should I trust any search engine crawler,) we could "load" our search engine with lists of websites. These lists are published and maintained by curators that we have explicitly chosen to trust. When we type into the search box it can…

You're re-inventing the blekko search engine.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#246

Post by salawat »

The problem isn't search engines per se.

The problem is closed algorithms, SEO, and advertising/marketing.

Think about it for a minute. Imagine a search engine that generates the same results for everyone. Since it gives the same results for everyone, the burden of looking for exactly what you're looking for is put back exactly where it needs to be, on the user.

The problem though, is you'll still get networks of "sink pages" that are optimized to show up in every conceivable search, that don't have anything to do with what you're searching for, but are just landing pages for links/ads.

Personally, I liked a more Yellow Pageish net. After you got a knack for picking out the SEO link sinks, and artificially disclose them, you were fine. I prefer this to a search provider doing it for you because it teaches you, the user, how to retrieve information better. This meant you were no longer dependant on someone else slurping up info on your browsing habits to try to made a guess at what you were looking for.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#247

Post by adrianmonk »

iblaine wrote: post #228

Yes, it was called Yahoo and it did a good job of cataloging the internet when hundreds of sites were added per week: https://web.archive.org/web/19961227005023/http://www2.yahoo... I'm old enough to remember sorting sites by new to see what new URLs were being created, and getting to that bottom of that list within a few minutes. Google and search was a natural response to solving that problem as the number of sites…

I used Yahoo back in those days, and it literally proved the point that hand-cataloging the internet wasn't tractable, at least not the way Yahoo tried to do it. There was just too much volume.

It was wonderful to have things so carefully organized, but it took months for them to add sites. Their backlog was enormous.

Their failure to keep up is basically what pushed people to an automated approach, i.e. the search engine.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#248

Post by politician »

Lately, I've been turning over an idea that in order to advance, the next generation of the Internet should be designed so that third-party advertising is impossible to implement. I believe that as a consequence, this requirement will prevent crawler-based search engines from operating which presents a source discovery problem.

Discovering new sources of information in this kind of environment is difficult, and basically boils down to another instance of the classic key distribution problem - out-of-band, word-of-mouth, and QR codes.

Search engines like Google and Bing solve the source discovery problem by presenting themselves as a single source; aggregating every other source through a combination of widespread copyright infringement and an opaque ranking algorithm.

Google and Bing used to do a great job of source discovery, but the quality of their results have deteriorated under relentless assaults from SEO and Wall Street.

I think it's time for another version of the Internet where Google is not the way that you reach the Internet (Chrome) or find what you're looking for on the Internet (Search) or how you pay for your web presence (Adsense).

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#249

Post by alangibson »

The 2 core flaws of the Internet (more precisely the World Wide Web) are lack of native search and native payments. Cryptocurrencies have started to address the second issue, but no one that I know of is seriously working on the first.

Fast information retrieval requires an index. A better formulation of the question might be: how do we maintain a shared, distributed index that won't be destroyed by bad actors.

I wonder if the two might have parts of the solution in common. Maybe using proof of work to impose a cost on adding something to the index. Or maybe a proof of work problem that is actually maintaining the index or executing searches on it.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#250

Post by ehnto »

ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

So long as there is a mechanism for categorizing information and ranking the results, people will try to game the mechanism to get the top spot regardless of your own incentives.

Despite their incentives to make money, Google have actually been trying for years to stop people from gaming the system. It's impressive how far they've been able to come, but their efforts are thrwarted at every turn thanks to the big budgets employed to get traffic to commercial websites.

Post reply on HN