Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

To Break Google’s Monopoly on Search, Make Its Index Public

bloomberg.com

Post reply on HN
Print view View original

61–70 of 630 posts

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#61

Post by noahl »

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible.

The main problem is, I think the author is wrong about what Google's "crown jewel" is. Yes, Google has a huge index, but most queries aren't in the long tail. Indexing the top billion pages or so won't take as long as people think.

The things that Google has that are truly unique are 1) a record of searches and user clicks for the past 20 years and 2) 20 years of experience fighting SEO spam. 1 is especially hard to beat, because that's presumably the data Google uses to optimize the parameters of its search algorithm. 2 seems doable, but would take a giant up-front investment for a new search engine to achieve. Bing had the money and persistence to make that investment, but how many others will?

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#62

Post by ailideex »

taf2 wrote: post #54

So although everyone likes to believe google is a monopoly it’s far from it. You have choices- bing, biadu , yandex, DuckDuckGo... there is also nothing about googles search position that prevents you from building a competitor. What we do have is peter thiel backing an administration that’s anti google, Russia, China that are anti google. Why? it’s a source of truth that challenges their lies. We also have an emerge…

> What we do have is peter thiel backing an administration that’s anti google, Russia, China that are anti google.

Can you provide some context? Article does not mention Thiel, Russia or China.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#63

Post by archy_ »

JumpCrisscross wrote: post #14
peteretep wrote: post #3

I don’t want to break Google’s monopoly on search. Google’s search is fantastic. It’s their advertising business knowing too much about me I care about.

> Google’s search is fantastic The author's proposal, making Google's index publicly accessible, would leave Google search intact. Google's algorithm would presumably remain proprietary. For people who like Google today, nothing would change. For people for whom Google falls short, there would be new options. Looks like a clear win-win for consumers.

>Google's algorithm would presumably remain proprietary.

This is not a good thing. Basic keyword searching on an index isn't viable to compete, then you go back to when keyword spammers were at the top. Google should be forced to distribute all their search technologies to the public with the threat of the Alphabet corporation being dissolved if they do not comply. That is the only way to get us out of this mess and break Google's stranglehold on the internet.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#64

Post by vgaldikas »

glitchc wrote: post #39
JumpCrisscross wrote: post #25

Earlier quoted context omitted.

> This idea is a pipe dream. It would be nationalizing property The author references precedent in the 1956 consent decree with Bell Labs [1]. If anything, that was more extreme. AT&T developed those patents. The author's proposal doesn't involve Google surrendering its algorithm. Just the index it compiled from public resources using a publicly-subsidized Internet infrastructure. All without content owners' permissi…

What about the computing and infrastructure resources that Google dedicated to build the index, and continues to dedicate to keep the index up to date? This is a failed model. There is no incentive for Google to continue to update the public version, and it would quickly fall out of date while Google focuses on their own internal copy. It’s a solution suggested by lawmakers who fundamentally don’t understand how comp…

Did you read the article?

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#65

Post by zubspace »

From the article:

"But what about those nasty filter bubbles that trap people in narrow worlds of information? Making Google’s index public doesn’t solve that problem, but it shrinks it to nonthreatening proportions. At the moment, it’s entirely up to Google to determine which bubble you’re in, which search suggestions you receive, and which search results appear at the top of the list; that’s the stuff of worldwide mind control. But with thousands of search platforms vying for your attention, the power is back in your hands. You pick your platform or platforms and shift to others when they draw your attention, as they will all be trying to do continuously."

But this is a huge problem. I'd rather have 10 independent search providers instead of 10 companies proxying the results of google. It's worse, if I don't even know from which index the results come from. I guess, many people don't know, that Startpage shows you Google results.

I don't want Google results! I want different web crawlers ordering the results according to my taste without tracking each and every page impression of me. Give me that and I'll switch in a heartbeat.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#66

Post by celticmusic »

taf2 wrote: post #54

So although everyone likes to believe google is a monopoly it’s far from it. You have choices- bing, biadu , yandex, DuckDuckGo... there is also nothing about googles search position that prevents you from building a competitor. What we do have is peter thiel backing an administration that’s anti google, Russia, China that are anti google. Why? it’s a source of truth that challenges their lies. We also have an emerge…

ah yes, the only reason you could ever have for being anti google, et al, is that you want to lie...

You started with a decent premise, that google isn't an actual monopoly because it has "competitors" (and yes, those quotes are intentional). But then you go off the deep end and completely lost me to the point that I didn't even finish the entirety of your post.

There are many many reasons that someone could be against google's absolute market dominance.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#67

Post by stereolambda »

sametmax wrote: post #46
JumpCrisscross wrote: post #14

Earlier quoted context omitted.

> Google’s search is fantastic The author's proposal, making Google's index publicly accessible, would leave Google search intact. Google's algorithm would presumably remain proprietary. For people who like Google today, nothing would change. For people for whom Google falls short, there would be new options. Looks like a clear win-win for consumers.

Like you said, the value of Google is the algo. The index is worth little. The limits of Bing, Qwant, DDG, etc. are not the number of the indexed pages. It's that, give them the same pages, and the same search terms, and they don't return results as good.

Maybe the practical indexing ability is worth more than the index itself. Most site owners happily let the Googlebot into their websites, while a competitor is likely to be caught into some sweeping anti-bot/Captcha measures.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#68

Post by [deleted] »

zubspace wrote: post #65

From the article: "But what about those nasty filter bubbles that trap people in narrow worlds of information? Making Google’s index public doesn’t solve that problem, but it shrinks it to nonthreatening proportions. At the moment, it’s entirely up to Google to determine which bubble you’re in, which search suggestions you receive, and which search results appear at the top of the list; that’s the stuff of worldwide…

[deleted]

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#69

Post by londons_explore »

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> 1) a record of searches and user clicks for the past 20 years

If a government was serious about getting more players in the search industry, they would force Google (and all other players) to make this data public.

Simply say "All user-behaviour data used to improve the service must be freely published".

Make the law apply to any web service with more than 20 million users globally so small businesses aren't burdened.

If the data cannot be published for privacy reasons, the private parts must be seperated and not used by google or it's competitors.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#70

Post by vntok »

JumpCrisscross wrote: post #50
sametmax wrote: post #46

Earlier quoted context omitted.

Like you said, the value of Google is the algo. The index is worth little. The limits of Bing, Qwant, DDG, etc. are not the number of the indexed pages. It's that, give them the same pages, and the same search terms, and they don't return results as good.

> the value of Google is the algo. The index is worth little. But it's worth something. Giving DuckDuckGo direct access to Google's index, including the ability to train models on said index, would improve the competitive landscape.

Or, you know, they could innovate by themselves instead of relying on Google to do all the heavy lifting?
Post reply on HN