Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can we create a new internet where search engines are irrelevant?

news.ycombinator.com

Post reply on HN
Print view View original

311–320 of 395 posts

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#311

Post by penagwin »

Retra wrote: post #278
adrianmonk wrote: post #257

I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…

Couldn't you just randomize result ordering?

You know how google search results can get really useless just a few pages in? And it says it found something crazy like 880,000 results? Imagine randomizing that.

---

Unrelated I searched for "Penguin exhibits in Michigan". Of which we have several. It reports 880,000 results but I can only go to page 12 (after telling it to show omitted results). Interesting...

https://www.google.com/search?q=penguin+exhibits+in+michigan

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#312

Post by z3t4 »

ovi256 wrote: post #110

Everyone has missed the most important aspect of search engines, from the point of view of their core function of information retrieval: they're the internet equivalent of a library index. Either you find a way to make information findable in a library without an index (how?!?) or you find a novel way to make a neutral search engine - one that provides as much value as Google but whose costs are paid in a different w…

A search engine is more like putting the books in a paper schredder and writing the book title on every piece, then ordering the pices by whatever words you can find on it, putting all pieces that has the word "hacker" on it in the same box. Where as the problem becomes how you sort the pieces. Want to find a book about "hacking"? This box has all the shreds that has the word "hacker" on it, you can find the book title on the back of the piece. Second problem becomes how relevant the word is to the book.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#313

Post by jka »

There'd be a feedback loop problem, but are DNS query logs a potential source of ranking/priority?

Over time the domains that users genuinely organically visit (potentially geo-localized based on client location) should rise in query volume.

Caveats would include DNS record cache times, lookups from robots/automated services, and no doubt a multitude of inconsistent client behavior oddities.

A similar approach could arguably be applied even at a network connection log level.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#314

Post by politician »

notahacker wrote: post #305
TeMPOraL wrote: post #168

Earlier quoted context omitted.

> How would you achieve that? By outrightly outlawing advertising? Yes. > There absolutely are paid options on the web. It's just that they don't seem to appeal to a sufficient number of buyers so advertising could become irrelevant. They aren't appealing in the presence of ad-subsidized free alternatives . Remove the latter, and they just might become appealing again.

Few things sound less likely to improve the internet than some entity having the power to content-police the web and remove anything it accuses of the thoughtcrime of advertising...

You can block third-party advertising structurally, so that a content-cop isn't required. First-party advertising cannot be blocked, of course, since that's just content.

For example, using browsers that impose a Content Security Policy that prevents anything from being loaded from domains other than the origin.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#315

Post by peteyPete »

You'd still want to be able to retrieve "useful" information which can't be tampered with easily which I think is the biggest issue.

You can't curate manually.. That just doesn't scale. You also can't let just anyone add to the index as they wish or any/every business will just flood the index with their products... There wouldn't be any difference between whitehat/blackhat marketing.

You also need to be able to discover new content when you seek it, based on relevancy and quality of content.

At the end of the day, people won't be storing the index of the net locally, and you also can't realistically query the entire net on demand. That would be an absolutely insane amount of wasted resources.

All comes back to some middleman taking on the responsibility (google,duckduckgo,etc).

Maybe the solution is an organization funded by all governments, completely transparent, where people who wish to can vote on decisions/direction. So non profit? Not driven by marketing?

But since when has government led with innovation and done so at a good pace? Money drives everything... And without a "useful" amount of marketing/ads etc, the whole web wouldn't be as it is.

So yes, you can.. But you won't have access to the same amount of data, as easily, will likely have a harder time finding relevant information (especially if its quite new) without having to parse through a lot of crap.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#317

Post by rhmw2b »

quelsolaar wrote: post #261

Yes, we need search engines, but they don't need to be monolithic. Imagine that indexing the text of your average web page takes up 10k. Then you get 100.000 pages per Gig. It means that you if you spend ~270USD on a consumer 10 tera drive you can index a billion webpages. Google no longer says how many pages they index, but its estimated to be with in one order of magnitude of that. This means that in terms of hardw…

Google's paper on Percolator from 2010 says there are more than 1T web pages. 9 years later there is surely way more than that.

https://ai.google/research/pubs/pub36726

The real issue would be crawling and indexing all those pages. How long would it take for an average user's computer with a 10Mb internet connection to crawl the entire web? It's not as easy a problem as you make it seem.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#318

Post by gist »

This is to broad a question to answer. There are really to many different uses of the Internet to try and fashion a solution that works in all areas. Not to mention the fact that it's to academic to begin with. How do you get such a large group of people to change a behavior that works for them already? And very generally most people are not bothered by the privacy aspect as much as tech people (always whining about things) are or even the media. People very generally like they can get things at no cost and don't (en masse) care anywhere near as much about being tracked as you have been led to believe. And that's when tracking is not even benefiting them which it is often. This is not 'how can we eliminate robo calls'. It's not even 'how can we eliminate spam'.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#319

Post by bouk »

alfanick wrote: post #108

I see a lot of good comments here, I got inspired to write this: What if this new Internet instead of using URI based on ownership (domains that belong to someone), would rely on topic? In examples: netv2://speakers/reviews/BW netv2://news/anti-trump netv2://news/pro-trump netv2://computer/engineering/react/i-like-it netv2://computer/engineering/electron/i-dont-like-it A publisher of webpage (same html/http) would pu…

If netv2 is neutral, I would just stuff all of the topics with my own content millions of time, so everyone can only see my content

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#320

Post by gexla »

KirinDave wrote: post #214
gexla wrote: post #169

Earlier quoted context omitted.

Twitter has a social graph and communities much like Reddit. This adds more information. There are people posting information who I trust and even know IRL. > All of those constraints are invisible. It's dangerous to think you have more control or insight there. And yet you are commenting as if these results aren't invisible to you? The machinery behind Google search results aren't invisible? Are you trying to say th…

> And yet you are commenting as if these results aren't invisible to you? The machinery behind Google search results aren't invisible? Because of my unique and fortunate work history I understand the internals of these systems better than many people do. I'm objecting to the distinction you're drawing, not suggesting an alternative order of transparency. There really isn't much difference between the two companies ou…

I agree that Twitter may not be much different from Google if you are relying on the algo. Notice that I mentioned Twitter as a community next to Reddit though. Everyone has different usage patterns for these services. I follow (and have been an active participant) in a number of niche communities on Twitter. In some cases, I could do a search for a term and most results would be from people I have interacted with through Twitter and other channels. Each of those people carried a reputation within that niche and some I knew better than others. I wouldn't use Twitter as a general search returning a large number of untrusted results. Sure, even search results on a specific user could be biased, but at least it would be from familiar territory.
Post reply on HN