Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

To Break Google’s Monopoly on Search, Make Its Index Public

bloomberg.com

Post reply on HN
Print view View original

101–110 of 630 posts

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#101

Post by NoGravitas »

peteretep wrote: post #3

I don’t want to break Google’s monopoly on search. Google’s search is fantastic. It’s their advertising business knowing too much about me I care about.

Google's search isn't even that great, to be honest. I use DDG for search, and use the !g operator when the DDG results aren't satisfying. But my experience with that is that the Google results are never good for those searches, either.

The thing the article misses is that search isn't really Google's crown jewel anymore. It's their position at the top of the adtech ecosystem, and while they bootstrapped that with search, I kind of doubt that search is the main thing driving their ad views today.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#102

Post by vanderZwan »

dkyc wrote: post #81
noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> Yes, Google has a huge index, but most queries aren't in the long tail. I'm not quite sure about that. 15% of Google searches per day are unique, as in, Google has never seen them before. [1]. That's quite an insane number. [1] https://searchengineland.com/google-reaffirms-15-searches-ne...

How many of those are confirmed to be of human origin?

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#103

Post by tryptophan »

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

>2) 20 years of experience fighting SEO spam.

Tangential - but does anyone else feel that google results are useless a lot of the time? If you search for something, you will get 100% SEO optimized shitty ad-ridden blog/commercial pages giving surface level info about what you searched about. I find for programming/IT topics its pretty good, but for other topics it is horrible. Unless you are very specific with your searches, "good" resources don't really percolate to the top. There isn't nearly enough filtering of "trash".

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#104

Post by rocgf »

dkyc wrote: post #81
noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> Yes, Google has a huge index, but most queries aren't in the long tail. I'm not quite sure about that. 15% of Google searches per day are unique, as in, Google has never seen them before. [1]. That's quite an insane number. [1] https://searchengineland.com/google-reaffirms-15-searches-ne...

Wow, 15% unique searches is indeed quite an interesting figure. With that said, what OP said is definitely not disproved. Just because 15% of searches are unique, that doesn't mean the most relevant result is buried in the tail end. I mean I can think of loads of my own searches that are probably unique or rare, but lead to the same popular results because of typos, improper wording etc.

Without some clear numbers on that from a major search engine, I think this might be very difficulty to infer.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#106

Post by nova22033 »

Caveat: The author is not a technologists

Robert Epstein (born June 19, 1953) is an American psychologist, professor, author, and journalist. He earned his Ph.D. in psychology at Harvard University in 1981, was editor in chief of Psychology Today,

He has also made some questionable claims about google manipulating search results to favor Hillary Clinton.

https://en.wikipedia.org/wiki/Robert_Epstein#cite_note-15

His research is based entirely on his own experience

“It is somewhat difficult to get the Google search bar to suggest negative searches related to Mrs. Clinton or to make any Clinton-related suggestions when one types a negative search term,” writes Dr. Robert Epstein, Senior Research Psychologist at the American Institute for Behavioral Research and Technology.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#107

Post by kyrra »

tyingq wrote: post #89
detritus wrote: post #78

Earlier quoted context omitted.

> Indexing the top billion pages or so won't take as long as people think. This is what makes me wonder why we don't have a LOT of competing search engines. Perhaps i'm vastly under-estimating the technology and difficulty (I could well be - it's not my domain) but it surely it can't be THAT hard to spawn Google-like weighted crawl-based search results? It's a long-since solved problem - heck, pageRank's first iterat…

SEO spam, and poor quality content I would guess. Google has bolted on a ton of ML over the last ten years to fight it.

It's not just ML, but the people that provide the labeling for the ML.

Google pays some large number of people to do search and grade the various results they get to see if the answers are good, which then helps feed back ML.

Heck, according to this article[0], google has been paying people to evaluate their search results since 2004.

[0] https://searchengineland.com/interview-google-search-quality...

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#109

Post by estebarb »

arch-ninja wrote: post #23

No sure if relevant but I've hated google's search for a unique reason: I find it horribly slow, especially compared to what it used to be. It's so slow, I've designed + partially implemented an alternative for my own use: https://github.com/Jeffrey-P-McAteer/dindex I've only tested with 1000 records, but the query times are all <200ms.

It's easy to achieve good performance on small datasets. But we are speaking of Internet scale!!! And not only that, Google is capable of serving responses to millions of persons!!!

The index take a lot of disk, memory, computing power, bandwidth... And then you have to handle attacks, spam, dead websites and not going bankrupt in the process.

I don't know you, but my best toy website is only capable of serving 100k reqs/second in localhost if serving pong...

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#110

Post by taf2 »

celticmusic wrote: post #66
taf2 wrote: post #54

So although everyone likes to believe google is a monopoly it’s far from it. You have choices- bing, biadu , yandex, DuckDuckGo... there is also nothing about googles search position that prevents you from building a competitor. What we do have is peter thiel backing an administration that’s anti google, Russia, China that are anti google. Why? it’s a source of truth that challenges their lies. We also have an emerge…

ah yes, the only reason you could ever have for being anti google, et al, is that you want to lie... You started with a decent premise, that google isn't an actual monopoly because it has "competitors" (and yes, those quotes are intentional). But then you go off the deep end and completely lost me to the point that I didn't even finish the entirety of your post. There are many many reasons that someone could be again…

Where do you see their absolute market dominance... I checked here for example http://gs.statcounter.com/ and for browsers globally it's 63% that's not a monopoly... I checked here https://www.statista.com/statistics/216573/worldwide-market-... for search engine and 90% is pretty close but it's not a monopoly... contrast that with MS in the late 90s according to https://www.forbes.com/sites/timworstall/2012/12/13/microsof... and you have 97% that's a monopoly if it's abused which it was proven they did.. https://en.wikipedia.org/wiki/United_States_v._Microsoft_Cor.... again i find it interesting it's hard to talk about google being a monopoly and not mentioning MS...
Post reply on HN