Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can we create a new internet where search engines are irrelevant?

news.ycombinator.com

Post reply on HN
Print view View original

321–330 of 395 posts

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#321

Post by notahacker »

politician wrote: post #314
notahacker wrote: post #305

Earlier quoted context omitted.

Few things sound less likely to improve the internet than some entity having the power to content-police the web and remove anything it accuses of the thoughtcrime of advertising...

You can block third-party advertising structurally, so that a content-cop isn't required. First-party advertising cannot be blocked, of course, since that's just content. For example, using browsers that impose a Content Security Policy that prevents anything from being loaded from domains other than the origin.

Sure, but if the only ad restriction was mandatory blocking of third party content, you'd just see ad agencies work out ways they can get the content they want to serve hosted locally (and lots of more interesting third party embedded content cease to exist due to it not having the same commercial rationale for workarounds...). If you start forcing companies not to promote third party products with anything that even looks like an ad, you'll just see a greater proportion of the free-to-access internet turn into paid-for reviews and influencer marketing. Not sure that'd be an improvement, and I'm pretty sure the next logical step of getting the content cops ruling which content looks too commercially-oriented for us proles to look at is even worse.

You can block third party advertising structurally using uBlock without ruining the internet for everyone else.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#322

Post by rglullis »

Fjolsvith wrote: post #94
rglullis wrote: post #54

Earlier quoted context omitted.

There will always be bottom-feeders as long as there is a market where people are not forced to choose with their wallets. Killing the "vast majority of content on the internet" seems like a good thing to me, honestly.

> Killing the "vast majority of content on the internet" seems like a good thing to me, honestly. I sure hope my content of preference beats out yours for not getting killed.

I am reasonably sure that even if our preferences are complete opposite and we eliminate 99% of content in general, you would still have enough quality content for what your interests are. But just to be extra sure, please vote with your wallet and actively support the things you like and don't let advertisers do the choosing for you.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#323

Post by jtolmar »

adrianmonk wrote: post #257

I think it would be helpful to remember to distinguish two separate search engine concepts here: indexing and ranking. Indexing isn't the source of problems. You can index in an objective manner. A new architecture for the web doesn't need to eliminate indexing. Ranking is where it gets controversial. When you rank, you pick winners and losers. Hopefully based on some useful metric, but the devil is in the details on…

Some thoughts on the problem, not intended as a complete proposal or argument:

* Indexing is expensive. If there's a shared public index, that'd make it a lot easier for people to try new ranking algorithms. Maybe the index can be built into the way the new internet works, like DNS or routing, so the cost is shared.

* How fast a ranking algorithm is depends on how the indexing is done. Is there some common set of features we could agree on that we'd want to build the shared index on? Any ranking that wants something not in the public index would need either a private index or a slow sequential crawl. Sometimes you could do a rough search using the public index and then re-rank by crawling the top N, so maybe the public index just needs to be good enough that some ranker can get the best result within the top 1000.

* Maybe the indexing servers execute the ranking algorithm? (An equation or SQL-like thing, not something written in a Turing Complete language). Then they might be able to examine the query to figure out where else in the network to look, or where to give up because the score will be too low.

* Maybe the way things are organized and indexed is influenced by the ranking algorithms used. If indexing servers are constantly receiving queries that split a certain way, they can cache / index / shard on that. This might make deciding what goes into a shared index easier.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#325

Post by quelsolaar »

rhmw2b wrote: post #317
quelsolaar wrote: post #261

Yes, we need search engines, but they don't need to be monolithic. Imagine that indexing the text of your average web page takes up 10k. Then you get 100.000 pages per Gig. It means that you if you spend ~270USD on a consumer 10 tera drive you can index a billion webpages. Google no longer says how many pages they index, but its estimated to be with in one order of magnitude of that. This means that in terms of hardw…

Google's paper on Percolator from 2010 says there are more than 1T web pages. 9 years later there is surely way more than that. https://ai.google/research/pubs/pub36726 The real issue would be crawling and indexing all those pages. How long would it take for an average user's computer with a 10Mb internet connection to crawl the entire web? It's not as easy a problem as you make it seem.

I'm not saying its easy, its not, but people tend to think that because Google is so huge, you have to be that huge to do what Google does. My argument is that in terms of hardware google need expensive hardware because they have so many users, not because what they do requires that hardware to deliver the service for one or a few users.

I have a gigabit link to my apartment (go Swedish infrastructure!). At that theoretic speed I get 450 gigs an hour, so I could download ten tera in a day. We can easily slow that down by an order of magnitude and its still a very viable thing to do. If someone wrote the software to do this, one could imagine some kind of federated solution for downloading the data, so that every user doesn't have to hit every web server.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#327

Post by asdff »

pif wrote: post #84
_nalply wrote: post #61

Earlier quoted context omitted.

Advertisement just should not be the central means of income of content producers. I really hope this point of view gets killed together with advertisement.

> Advertisement just should not be the central means of income of content producers. Can you propose any viable alternative?

Certain % of your internet bill goes to helping pay to host the sites you are visiting every billing period. If a site is large enough hosting would be sustained by the visiting userbase rather than the site owner. If a site is too small for that, chances are hosting has been cheap anyway.

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#328

Post by volkk »

WhompingWindows wrote: post #175
alfanick wrote: post #108

I see a lot of good comments here, I got inspired to write this: What if this new Internet instead of using URI based on ownership (domains that belong to someone), would rely on topic? In examples: netv2://speakers/reviews/BW netv2://news/anti-trump netv2://news/pro-trump netv2://computer/engineering/react/i-like-it netv2://computer/engineering/electron/i-dont-like-it A publisher of webpage (same html/http) would pu…

You want to silo information and create built-in information echo chambers? That seems so bad for polarization.

im starting to think echo chambers are just something that will forever be prevalent and its up to the users to try to view alternate viewpoints

Re: Ask HN: Can we create a new internet where search engines are irrelevant?

#329

Post by weliketocode »

Your two points really don’t fit with your follow-up explanation.

If you don’t believe finding information is currently trivial using Google, that’s going to be a tough nut to crack.

What would you use for information retrieval that doesn’t involve indexing or a search engine?

Post reply on HN