To Break Google’s Monopoly on Search, Make Its Index Public • Hacker News Board

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#431

Post by kingludite » Mon, Jul 15, 2019, 6:48 PM UTC

nojvek wrote: post #355

com2kid wrote: post #345

Earlier quoted context omitted.

> 1) a record of searches and user clicks for the past 20 years From what I can tell, Google cares a lot more about recency. When I switch over to a new framework or language, search results are pretty bad for the first week, horrible actually as Google thinks I am still using /other language/. I have to keep appending the language / framework name to my queries. After a week or so? The results are pure magic. I can…

Going back to OPs point. Google is real good at associating search query to search result. Every time you search and click on something, google learns that association. So it could very well be that as more users adopt the new language/framework in the first couple of weeks they have taught google those associations. Google isn’t a search company. They are a distributed machine learning company that make most of thei…

They have adds to show first, telling what people want comes after that, knowing what people wanted is only to make the second easier and only interesting up to serving the first.

Really good or really bad only exists if there is something else to compare it to.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#432

Post by mrtksn » Mon, Jul 15, 2019, 6:48 PM UTC

nostrademons wrote: post #400

mrtksn wrote: post #388

Earlier quoted context omitted.

I think it is possible to make way, way better search engine because Google Search is no longer as good as it used to, at least for me. I can no longer find anything remotely good quality, I discover new and quality stuff from social media like Twitter and HN. The search results seem to be too general and too mainstream. Nothing new to discover, just a shortcut to the few websites like Reddit, StackOverflow for more…

The reason for that is because Google's building for a mainstream audience, because the mainstream (by definition) is much bigger than any niche. They increase aggregate happiness (though not your specific happiness) a lot more by doing so. It's probably possible to build a search engine for a specific vertical that's better than Google. However, you face a few really big problems that make this not worthwhile: 1) Sp…

Sure, they have a business reason to do exactly what they do but I think as people grow up they specialize and the general stuff that fits everybody becomes useless. Google tries to personalize search results but that so far yielded echo chambers, not personalized discoveries.

I can't get better products by searching Google, I can get the best-spammed products or most promoted products only.

The fact that I am getting low-quality service and Google is printing money means that there is a place for good a good service and if that service cannot emerge due to Google's practices, it probably means that the regulators need to take action.

Or maybe the search is dead, long live social media.

The gist is, I am not happy with a service but the company that makes that product makes a lot of maney. Can't tell if I am an anomaly or if other people feel the same way because Google is a monopoly and maybe the regulators should make it possible to compete with Google and see if there's a space for a better service.

Yes yes, I am the product but I am the product only if I am happy with the stuff I'm getting in return.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#433

Post by malvosenior » Mon, Jul 15, 2019, 6:51 PM UTC

bduerst wrote: post #429

malvosenior wrote: post #422

Earlier quoted context omitted.

> Now, if you suspect that the instant search results are politically biased, then the burden of proof is on you to provide evidence of that existence The proof is a senior Google employee admitting to bias and manipulating results in the Project Veritas video. There's also plenty of anecdotal evidence you can see for yourself as a user. In addition to that I know many people who work at Google and the vast majority…

This is precisely the hand-waving I was talking about. An employee rambling in a bar does not constitute evidence of search result manipulation, especially when the person recording it is known for stretching information, inciting people to commit voter fraud, and crossing the U.S. Mexico border dressed as deceased Osama Bin Laden to prove some point. Likewise, the handful of people you know having political views, i…

You asked for proof and I gave it to you. I'm sorry you don't like the source, but unless you want to address what Jen Gennai said, I don't think you have much of a point.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#434

Post by smsm42 » Mon, Jul 15, 2019, 6:52 PM UTC

com2kid wrote: post #345

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> 1) a record of searches and user clicks for the past 20 years From what I can tell, Google cares a lot more about recency. When I switch over to a new framework or language, search results are pretty bad for the first week, horrible actually as Google thinks I am still using /other language/. I have to keep appending the language / framework name to my queries. After a week or so? The results are pure magic. I can…

I was curious about this, as I work in multiple languages every day. I almost never use Google though except as last resort if other engines can't find anything. So the result I got for array length was for Javascript. Which is quite high on the hype cycle now, but I only very rarely use it and search anything about it even less frequently.

Sо I wonder how much of the magic you perceive might be just your interests matching the interests of the most other people using Google and thus it's not just Google magically guessing you're into Javascript (for example) now, but Javascript being popular and this being the cause of both Google returning matches for it and you starting to use it? Did you ever do a clean experiment - e.g. try to learn APL or some other relatively obscure language and have Google return all results about APL and none about Javascript?

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#435

Post by mtnGoat » Mon, Jul 15, 2019, 6:53 PM UTC

phh wrote: post #71

taf2 wrote: post #54

So although everyone likes to believe google is a monopoly it’s far from it. You have choices- bing, biadu , yandex, DuckDuckGo... there is also nothing about googles search position that prevents you from building a competitor. What we do have is peter thiel backing an administration that’s anti google, Russia, China that are anti google. Why? it’s a source of truth that challenges their lies. We also have an emerge…

Carousel requiring (severely favoring?) AMP, requiring exclusively Google javascript/ads is definitely Googling using their advanced user base on search to favor their ad business. They have already been condemned for their Google Shopping service for monopoly abuse on search.

condemned only in the EU and the solution that was created did NOT help the consumer.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#436

Post by majormajor » Mon, Jul 15, 2019, 6:54 PM UTC

com2kid wrote: post #345

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> 1) a record of searches and user clicks for the past 20 years From what I can tell, Google cares a lot more about recency. When I switch over to a new framework or language, search results are pretty bad for the first week, horrible actually as Google thinks I am still using /other language/. I have to keep appending the language / framework name to my queries. After a week or so? The results are pure magic. I can…

I always see posts like this here, and then I try it, and I get a page full of "array length" results for Javascript, while everything in the last year that I've searched for has been Java or Kotlin...

Same when I owned a Pixel after hearing about Google Now and their ML magic there. Nothing more magical than an iPhone in terms of suggestions. The camera was amazing, but not all this supposed contextual stuff.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#437

Post by pavs » Mon, Jul 15, 2019, 6:54 PM UTC

nostrademons wrote: post #403

parentheses wrote: post #385

Earlier quoted context omitted.

I don't necessarily agree. The hard part of search is building the index and differentiating _real_ promotion from the _fake_. There's a lot of SEO manipulation that Google does a good job avoiding.

Webspam is a really big problem, yes. It's very unlikely that you'd be able to catch up or keep up in that regard without Google's resources. Building the index itself is relatively easy. There are some subtleties that most people don't think about (eg. dupe detection and redirects are surprisingly complicated, and CJK segmentation is a pre-req for tokenizing), but things like tokenizing, building posting lists, and…

I am not even that good of a programmer and I also agree with you that index relatively trivial. Other major issues, besides fighting spam:

- Hardware infrastructure and data center presence for extremely fast search from anywhere in the world. - Near real-time search suggestion. - personalized search results based on past search + geolocation. - Search to get instant results without having to go to a website.

Just to name a few. Google Search is the gold standard of a search engine, not because its Google or because they have been around for a long time and the brand name sticks (I am sure it helps too), but for the simple fact is no search engine is even remotely close to being as good as google and I have tried them all more of the less and given them shot. They are just not good at all.

I also don't understand the hate towards google being in charge of so many products so many people use, ie, Mail, Maps, Chrome, Android, Docs (to name a few). It's simply because they are damn good at it. If its a crime to make a product so good that people continue to use it, then I don't know what else people are supposed to do. As if we are asking google to make shit products, I just don't understand the reasoning.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#438

Post by feanaro » Mon, Jul 15, 2019, 6:57 PM UTC

com2kid wrote: post #345

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

> 1) a record of searches and user clicks for the past 20 years From what I can tell, Google cares a lot more about recency. When I switch over to a new framework or language, search results are pretty bad for the first week, horrible actually as Google thinks I am still using /other language/. I have to keep appending the language / framework name to my queries. After a week or so? The results are pure magic. I can…

The flip side of this is that it makes it harder for you to stumble upon something related, but new, outside of the filter bubble Google is making for you.

There's no arguing what you're describing is useful, but it's nice to keep in mind that there are downsides even if you ignore the privacy argument (which, IMO, shouldn't be ignored).

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#439

Post by retpirato » Mon, Jul 15, 2019, 6:58 PM UTC

Regardless of its search index or anything else, Google's dominance really comes more from it amassing a large userbase from the early days of the internet when no other search engine came close than it does from its search index.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#440

Post by Ivoirians » Mon, Jul 15, 2019, 6:59 PM UTC

briandear wrote: post #138

nova22033 wrote: post #106

Caveat: The author is not a technologists Robert Epstein (born June 19, 1953) is an American psychologist, professor, author, and journalist. He earned his Ph.D. in psychology at Harvard University in 1981, was editor in chief of Psychology Today, He has also made some questionable claims about google manipulating search results to favor Hillary Clinton. https://en.wikipedia.org/wiki/Robert_Epstein#cite_note-15 His r…

> He has also made some questionable claims about google manipulating search results to favor Hillary Clinton. Despite it being off topic, can we define why those claims are questionable? Is their data proving those claims wrong? Because with all the Google political controversies over the past few years, and given the political donation history of Google employees, it’s highly plausible that search results are manip…

Disclaimer: Googler.

I'm going to retread a comment I made when I first learned about Epstein a few months ago: https://news.ycombinator.com/item?id=19167084

In short, and ignoring my ad hominem attack on his motivations, I encourage you to read/skim his two "studies" [1][2] and see how absurd they are. You might dismiss my claims and summaries as biased, but I think I was pretty open-minded towards his conclusions until I read them.

[1] https://www.pnas.org/content/pnas/112/33/E4512.full.pdf?with...

[2] https://aibrt.org/downloads/EPSTEIN_&_ROBERTSON_2017-A_Metho...