Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

To Break Google’s Monopoly on Search, Make Its Index Public

bloomberg.com

Post reply on HN
Print view View original

381–390 of 630 posts

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#381

Post by bllguo »

fouc wrote: post #49
peteretep wrote: post #3

I don’t want to break Google’s monopoly on search. Google’s search is fantastic. It’s their advertising business knowing too much about me I care about.

I think the better move is to find a way to kill the advertising business model that google relies on. If there's no advertising there's not as much need for all this data collection.

the advertising model google relies on is just a reflection of how the attention economy and the internet work. I just don't see how this makes sense practically. People aren't going to seamlessly and happily go back to paying $$ for internet services

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#382

Post by ar-jan »

root_axis wrote: post #330
bodono wrote: post #236

Earlier quoted context omitted.

Try "hillary clinton emai". From a fresh chrome session in NYC I get nothing, not a single autocomplete result. On the other hand "donald trump emai" gets: * donald trump email * donald trump email address * donald trump email list * donald trump email newsletter * donald trump email list signup And just to drive the point home I tried "root_axis emai" and got "root_axis email". Try anyone else and you get similar re…

When I type "Donald Trump R" (or Ru or Russ etc) no autocomplete results contain the word Russia despite plenty of news coverage. When I type "Donald Trump Epstein" no autocomplete results. Must be a conspiracy by google to protect the president... or drawing conclusions based on individually cherry-picked autocomplete results is like drawing conclusions from numerology.

For me "donald trump ru" autosuggests "russia investigation" (4th place). Compared with absence of "email" even for "hillary clinton emai".

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#384

Post by bllguo »

peteretep wrote: post #3

I don’t want to break Google’s monopoly on search. Google’s search is fantastic. It’s their advertising business knowing too much about me I care about.

but a big reason search is fantastic is because of the data the system has on us. I don't think you can separate the two.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#385

Post by parentheses »

nojvek wrote: post #361
nostrademons wrote: post #309

Ex-Google-Search engineer here, having also done some projects since leaving that involve data-mining publicly-available web documents. This proposal won't do very much. Indexing is the (relatively) easy part of building a search engine. CommonCrawl already indexes the top 3B+ pages on the web and makes it freely available on AWS. It costs about $50 to grep over it, $800 or so to run a moderately complex Hadoop job.…

This ^ times a 1000. Google simply has the best search product. They invest in it like crazy. I’ve tried bing multiple times. It’s slow, it spams msn ads in your face on the homepage. Microsoft just doesn’t get the value of a clean UX. DuckDuckGo results are pretty irrelevant the last time I tried them. There is nothing that comes close to their usability. To make the switchover, it has to be much much better than Go…

I don't necessarily agree. The hard part of search is building the index and differentiating _real_ promotion from the _fake_. There's a lot of SEO manipulation that Google does a good job avoiding.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#386

Post by malvosenior »

bduerst wrote: post #370
briandear wrote: post #138

Earlier quoted context omitted.

> He has also made some questionable claims about google manipulating search results to favor Hillary Clinton. Despite it being off topic, can we define why those claims are questionable? Is their data proving those claims wrong? Because with all the Google political controversies over the past few years, and given the political donation history of Google employees, it’s highly plausible that search results are manip…

The "Manipulating instant search results in favor of Hillary Clinton" claim has been independently debunked and anyone still standing behind it are only signalling their technical illiteracy and/or political agenda for playing a victim card. [1][2][3] That's not really off-topic - The fact that the author still supports the claim calls into question their ability to make further claims about the subject. [1] https://…

None of those blog posts debunk the idea that Google manipulates search results to favor particular political parties. Mashable has a statement from Google (the other two don't), saying that they don't but why would they say if they were?

None of these debunk the claims made by Google employees in the Project Veritas videos.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#387

Post by inlined »

caust1c wrote: post #303
madez wrote: post #287

Earlier quoted context omitted.

> 15% of Google searches per day are unique, as in, Google has never seen them before. That is impossible, and therefore wrong (I'm wrong, please see below). To know if a search is unique, as in Google has never seen them before, Google must be able to decide if a query it receives was seen before or not. Even if we assume Google needed only one bit for each message it has ever seen, and assuming it only saw 15% of n…

This is somewhat a faulty analysis. One could easily use a high accuracy bloom filter to store whether a search has definitely not been seen before, and that would be an estimate on the lower bound of the error margin.

Yup. This was actually an interview question I got from a former Google search engineer.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#388

Post by mrtksn »

nostrademons wrote: post #309

Ex-Google-Search engineer here, having also done some projects since leaving that involve data-mining publicly-available web documents. This proposal won't do very much. Indexing is the (relatively) easy part of building a search engine. CommonCrawl already indexes the top 3B+ pages on the web and makes it freely available on AWS. It costs about $50 to grep over it, $800 or so to run a moderately complex Hadoop job.…

I think it is possible to make way, way better search engine because Google Search is no longer as good as it used to, at least for me.

I can no longer find anything remotely good quality, I discover new and quality stuff from social media like Twitter and HN.

The search results seem to be too general and too mainstream. Nothing new to discover, just a shortcut to the few websites like Reddit, StackOverflow for more techie thing and Wikipedia and the few mainstream news websites for the rest.

I usually end up to search HN, Reddit or StackOverflow directly as the resulting quality is better as I can get easily specific. Getting specific is harder on Google because it just omits or misinterprets my search query keywords quite often.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#389

Post by bduerst »

malvosenior wrote: post #386
bduerst wrote: post #370

Earlier quoted context omitted.

The "Manipulating instant search results in favor of Hillary Clinton" claim has been independently debunked and anyone still standing behind it are only signalling their technical illiteracy and/or political agenda for playing a victim card. [1][2][3] That's not really off-topic - The fact that the author still supports the claim calls into question their ability to make further claims about the subject. [1] https://…

None of those blog posts debunk the idea that Google manipulates search results to favor particular political parties. Mashable has a statement from Google (the other two don't), saying that they don't but why would they say if they were? None of these debunk the claims made by Google employees in the Project Veritas videos.

Except you're wrong in that you can't logically prove something doesn't exist. You can't prove that pink unicorns don't exist, just as much as you can't prove that political bias in the search results don't exist.

All you can do is disprove the claims of their existence. Someone tries to claim there's a pink unicorn in the garage, and you can check the garage and say that the it is pink unicorn free. Someone tries to claim political bias exist for hillary clinton in instant search results, and these articles disprove that claim as using cherry picked evidence.

Now, if you suspect that the instant search results are politically biased, then the burden of proof is on you to provide evidence of that existence - preferrably without cherry picking evidence to fit an agenda yourself. Otherwise it's just hand waving and click bait.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#390

Post by ChuckMcM »

noahl wrote: post #61

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…

Pretty much, and the potential for criminal activity is astronomical if you give them access to an open index. Things like every website on the web hit with the same zero day on the same day for maximum profit. Build your own best kiddie pron site evah! with direct access to the index and your own ranking system. What your admin pushed a config that left the admin pages open? Go time!

As someone who was operationally responsible for a search index (formerly VP Ops at Blekko) the kinds of things crooks tried to do was pretty instructive on how they use search in advancing their efforts.

Post reply on HN