To Break Google’s Monopoly on Search, Make Its Index Public • Hacker News Board

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#281

Post by bin0 » Mon, Jul 15, 2019, 3:39 PM UTC

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

More importantly, Google's core competency is PageRank. Sharing the index != sharing PageRank. As time goes on, others will use inferior algorithms, and become worse. This scheme will not accomplish what it intends to do. Also, you can't just force people to give away their property.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#282

Post by dalbasal » Mon, Jul 15, 2019, 3:40 PM UTC

empath75 wrote: post #228

dalbasal wrote: post #211

So... this article is a good example of how ??!? it gets once you move from "We gotta do something about these tech monopolies" into the "what should we do?" phase. How exactly "do* you break up a Google or a FB so that (only one possible reason, but the one cited here) they don't control too much media/mind share?^ Laws usually want to be general, and my suggestion doesn't necessarily lend to that, but I'll suggest…

> How exactly "do* you break up a Google or a FB so that (only one possible reason, but the one cited here) they don't control too much media/mind share?^ The problem isn't so much that google has a monopoly on search as that they abuse the monopoly to force their way into new businesses. Force google to spin search off as a separate entity from the rest of the company.

Isn't it? I'm not even sure that "monopoly" is the right frame. Google's "price" in their high market share areas (eg search) is free anyway.

To me, the problem is power, not necessarily market/pricing power or even exclusion power. The problem is that the "public square" is essentially a handful of private monopolies. What Google considers "inappropriate" is banned or disadvantaged on YouTube. What FB (or Twitter) considers distasteful can become absent from the public or political debate.

It's no joke, at this point. Things "woke up" when it seemed that shady stuff happened during trump's election. But.. several revolutions had already started on social media, coups, counter-coups, civil wars. 10s of thousands of public officials around the world have been elected on FB.

That is a lot of power in very few hands, under a veil of secrecy, often with bad incentives. It was cheap printed paphlets and union halls that made fascists-vs-communist street fights a thing in the 30s... mediums. These days, an algorithm notices that telling (proverbial) fascists about communists is good for engagement... And the algorithm optimizes. I'm not accusing them of doing this deliberately doing this, but the feed algorithms optimize this way for a reason.. incentive.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#283

Post by martin_drapeau » Mon, Jul 15, 2019, 3:40 PM UTC

The article mentions a precedent: "in the 1956 consent decree in the U.S. in which AT&T agreed to share all its patents with other companies free of charge."

Today the picture is completely different. Google, Amazon, Facebook and Apple are global - their infrastructure is virtual - not physical lines and cables. If a government wants to cause harm to one of those companies, they can decide to "move" somewhere else.

In addition GAFA is capturing dollars from everywhere in the world and bringing it to the US. It's in the US governments interest to tread softly here. No matter how the media spins things, GAFA is great for the US economy.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#284

Post by echelon » Mon, Jul 15, 2019, 3:42 PM UTC

epicide wrote: post #136

samscully wrote: post #93

Earlier quoted context omitted.

The ad business would pay the search business for ad placement. The search business would also be free to auction ad space to other providers.

> The search business would also be free to auction ad space to other providers. Doesn't that make them an ad business?

It makes them like the New York Times, which sells ad real estate. It's a much different business.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#285

Post by bambax » Mon, Jul 15, 2019, 3:42 PM UTC

ngngngng wrote: post #199

tantalor wrote: post #172

Earlier quoted context omitted.

> but with Google Analytics... No: "Search does not use Google Analytics for ranking" https://www.youtube.com/watch?v=LLmO1GE4GvI

I think you misunderstand what I'm saying. I'm not saying Google analytics will get you ranked higher or lower. Just that Google can use the data from analytics to tell if their results were what the user wanted.

It doesn't even have to link the results to a specific user, linking them to the search terms is enough.

It's possible to achieve something similar by measuring bounce rate, but in any case, the sheer number of searches is what matters. For new entrants, this kind of expertise will be very hard to get.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#286

Post by learnfromstory » Mon, Jul 15, 2019, 3:42 PM UTC

packet_nerd wrote: post #226

tryptophan wrote: post #103

Earlier quoted context omitted.

>2) 20 years of experience fighting SEO spam. Tangential - but does anyone else feel that google results are useless a lot of the time? If you search for something, you will get 100% SEO optimized shitty ad-ridden blog/commercial pages giving surface level info about what you searched about. I find for programming/IT topics its pretty good, but for other topics it is horrible. Unless you are very specific with your s…

Someone linked to an interesting site talking about how to make homemade hot sauce here on HNs. I partly read it and thought it was a great clean site and something I wanted to try. Later going back to find it again I literally spent hours searching, even though I'm pretty sure I remembered some of the exact phrases. For some reason recipe related search results are really really terrible on both Google and Bing.

Sometimes sites get dropped from the results because they are malware hosts. It’s more likely to happen to small independent sites. They are also more likely to just pack it up and shut down their sites.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#287

Post by madez » Mon, Jul 15, 2019, 3:43 PM UTC

dkyc wrote: post #81

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> Yes, Google has a huge index, but most queries aren't in the long tail. I'm not quite sure about that. 15% of Google searches per day are unique, as in, Google has never seen them before. [1]. That's quite an insane number. [1] https://searchengineland.com/google-reaffirms-15-searches-ne...

> 15% of Google searches per day are unique, as in, Google has never seen them before.

That is impossible, and therefore wrong (I'm wrong, please see below). To know if a search is unique, as in Google has never seen them before, Google must be able to decide if a query it receives was seen before or not. Even if we assume Google needed only one bit for each message it has ever seen, and assuming it only saw 15% of new messages each day since its creation more than 20 years ago, it would need to store more than 2^1471 bits.

What could be true is that each day 15% of all searches are unique on that day.

Edit: I'm wrong. The 15% of completely unique messages per day are in regards to the messages per day, and not in regards to all messages it has ever seen, therefore exponential growth doesn't apply. To see that, assume Google just received one search query each day for 20 years but it was unique random gibberish, then Google could easily save that even though 100% of all messages per day are unique.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#288

Post by samnwa » Mon, Jul 15, 2019, 3:43 PM UTC

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

It is the crown jewel because people choose Google precisely because they are understood to have the largest index. It's comparable to Verizon marketing 'the largest network,' but with many more benefits accrued to the company who is believed to have the largest search index.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#289

Post by scott_s » Mon, Jul 15, 2019, 3:43 PM UTC

bodono wrote: post #236

root_axis wrote: post #209

Earlier quoted context omitted.

This is not "research" period. Using one arbitrary search comparison to draw conclusions about the nature of a system that processes billions of queries a day is pretty weak. Additionally, I don't get the same results you do. "hillary clinton e" does not bring up emails, nor does "donald trump e" bring up emails (the first results I see are election, education, england visit, ex wife). I'm not ruling out the possibil…

Try "hillary clinton emai". From a fresh chrome session in NYC I get nothing, not a single autocomplete result. On the other hand "donald trump emai" gets: * donald trump email * donald trump email address * donald trump email list * donald trump email newsletter * donald trump email list signup And just to drive the point home I tried "root_axis emai" and got "root_axis email". Try anyone else and you get similar re…

I got curious and tried the names of a bunch of public figures. Some of " e" yielded "email" as the suggestion. But these did not: elizabeth holmes, tom jones, tom cruise, brad pitt, gwyneth paltrow, roger federer, will smith, jimmy carter.

Since Hillary Clinton is not unique, then it's not proof that her results are treated differently.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#290

Post by IronWolve » Mon, Jul 15, 2019, 3:44 PM UTC

Just "Google it" means to find an answer, if that answer is being changed by a megacorp for unknown and sometimes nefarious reasons, I'd say we have a problem.

Would you want to use a library that filters its books to push an ideology? Would you want to listen to the radio if they only pushed top 40? Would you want to have a doctoral thesis that needs corporate approval? Research books that exclude facts based on corporate profits?