Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Google so hysterically hypocritical about Bing using its public data?

roughlydrafted.com

Post reply on HN
Print view View original

91–100 of 133 posts

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#91

Post by haberman »

patio11 wrote: post #40
Joakal wrote: post #14

http://www.google.com/robots.txt First two lines: User-agent: * Disallow: /search

There is no robot involved. An actual human instructs their user agent to retrieve a document on Google. A piece of code, running in that user agent, does something. If anyone has a complaint here, it is the user, not Google. (Google really really doesn't want to spin this as "MS is spying on our users" because as soon as they say that some person is going to point out "Google is spying on the entire world" and that…

By that logic IE could send the entire page of search results back to Microsoft and it wouldn't be a violation of robots.txt. Because hey, it was a user-agent doing it, not a robot! If this is so in the clear, why not have Microsoft do that also?

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#92

Post by newt »

luigi wrote: post #41
greyman wrote: post #24

Earlier quoted context omitted.

Microsoft do NOT steal Google's search results. They use clickstream data from Bing toolbar users to improve their search results. What happened is that Google engineers engineered a use case, in which signal from those clickstream appear like a stealing of search results. Or in other words, Microsoft uses Google users behaviour in their ranking algorithm. But that's not unethical, and Google is doing the same thing.

But the effect is the same as intentional scraping and outright stealing: search results that could have only come from Google are appearing as Bing results. Bing needs to blacklist Google from its clickstream. Simple.

the effect is the same as intentional scraping and outright stealing

The google engineers intentionally sent this click data to Bing, so is Bing really stealing? It's odd to act surprised when Bing uses the data that was intentionally sent to it. Bing could specifically ignore Google search results pages when it is tracking clicks, but is that legitimate? Google scrapes everything, why shouldn't Bing?

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#93

Post by ddemchuk »

The world's largest scraper is complaining about being scraped...when Yelp and all the rest of the local businesses directories realized they were being scraped and the data was being used to google's advantage, it doesn't appear that Google gave two shits. Tables have turned, and it's like someone stole the handball from the recess yard.

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#94

Post by Dylan16807 »

motti wrote: post #80
patio11 wrote: post #40

Earlier quoted context omitted.

There is no robot involved. An actual human instructs their user agent to retrieve a document on Google. A piece of code, running in that user agent, does something. If anyone has a complaint here, it is the user, not Google. (Google really really doesn't want to spin this as "MS is spying on our users" because as soon as they say that some person is going to point out "Google is spying on the entire world" and that…

Although strictly speaking you're right, it breaks the intent of robots.txt as the users' clickstream is fed into the server-side indexing engine alongside robot crawled data. Arguing that according to the strict letter of the spec, robots.txt only has to be obeyed by true crawlers does hold water as strictly speaking Bing could disregard robots.txt altogether - it's not certainly not legally enforceable. The intent…

If I'm google and I block /search I want no company to collect the data.

If I'm a site that blocks /dynamic because I want to make sure my stats are accurate, not factoring bots, and I want to keep the load down on slow-generating pages, then I'm perfectly happy with the data being collected.

Is it the intent of robots.txt itself to block the clickstream data, or is it just google's intent?

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#95

Post by moultano »

dekz wrote: post #30
greyman wrote: post #24

Earlier quoted context omitted.

Microsoft do NOT steal Google's search results. They use clickstream data from Bing toolbar users to improve their search results. What happened is that Google engineers engineered a use case, in which signal from those clickstream appear like a stealing of search results. Or in other words, Microsoft uses Google users behaviour in their ranking algorithm. But that's not unethical, and Google is doing the same thing.

Microsoft uses behaviour I doubt they are only targeting Google. But I completely agree, the data is not being mined from Google, it's a relationship for search term a to page b created by the user, not stolen from Google.

Google is the one creating the association, not the user. The relationship between term a and page b was created by Google, and confirmed by a user. People don't click on search results that aren't there.

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#96

Post by johnjhayes »

jonburs wrote: post #22
axod wrote: post #7

Earlier quoted context omitted.

Also websites can opt-out of having their content indexed. How can Google opt-out of having its search index copied by Microsoft? The funniest argument in this has been "Bing hasn't got the data, Google has an unfair advantage etc" as if Microsoft didn't exist before Google. Microsoft have had years headstart to get their online division doing something worthwhile, but they still can't get it right.

No one has shown any evidence that Microsoft has directly copied Google's search index, or even violated the directives in google.com's robots.txt file. What Microsoft apparently has done is to infer relationships from Gooogle SRPs to various sites by way of information gathered through the Bing toolbar (and possibly other documented and user opt-in channels w/in IE). Using toolbar collected signals is well-establish…

The only thing I haven't seen is this. Ok, using the clickstream data is an established practice, but does Google use clickstream data from users making searches on OTHER search engines, or just their own?

This is the crux for me, Bing is using data from Google, does Google use data from Bing or only themselves?

Even if all true, I don't see it as unethical business practice. Definitely a marketing black eye, but not much else.

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#97

Post by anonymous246 »

nailer wrote: post #51
Encosia wrote: post #47

Earlier quoted context omitted.

The Google engineers who planted the phrases were running the Bing toolbar themselves. That part isn't in question.

So MS are saying Google themselves uploaded the data to MS by using Bing toolbar? Does Google agree their engineers did this?

Dude, I've seen your comments on this thread. Several of them are basically asking people to confirm facts everybody agrees on. You should come better prepared for the next discussion.

Sorry to be an ass. But you're wasting my time on this site, since I have to wade through your questions to get to the interesting ones.

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#98

Post by nlogn »

luigi wrote: post #41
greyman wrote: post #24

Earlier quoted context omitted.

Microsoft do NOT steal Google's search results. They use clickstream data from Bing toolbar users to improve their search results. What happened is that Google engineers engineered a use case, in which signal from those clickstream appear like a stealing of search results. Or in other words, Microsoft uses Google users behaviour in their ranking algorithm. But that's not unethical, and Google is doing the same thing.

But the effect is the same as intentional scraping and outright stealing: search results that could have only come from Google are appearing as Bing results. Bing needs to blacklist Google from its clickstream. Simple.

I don't even think they should just blacklist Google. They should just respect robots.txt.

edit: I should have clarified. I know that the Bing crawler likely respects robots.txt, but if they are using clickstream info to build their index, it seems right that they should respect robots.txt there as well, no?

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#99

Post by tungwaiyip »

I'll try to make my point again. There is one school of thinking that Google is profiting by farming information from the Internet, literally on the back of many people's labor. You can prove this by running a "Google Sting", that is known as "Google bombing". By having many people setting up honeypot to link an arbitrary keyword to certain URL, you seem to be able to fool Google to artifically associate such keyword with the URL, thus "proving" Google is profiting from your labor. I don't really buy this logic. But Google's accusation of Bing steal from them seems very close to this line of framing.

Re: Why is Google so hysterically hypocritical about Bing using its public data?

#100

Post by beoba »

nhebb wrote: post #44
InclinedPlane wrote: post #6

Earlier quoted context omitted.

Explain what you mean when you say that Google makes money by aggregating other people's content? Google makes money by indexing other people's content and driving traffic to other people's sites. That hardly seems like a bad thing.

http://books.google.com is considered one of the more egregious examples, with many authors and publicists having complained about Google taking samples of the content and posting it online. I did a Google search a while back on a graphics issue, and got the answer I needed from a Google book search result. You could argue (as Google does) that they are really just helping promote the book. But I have no need too buy…

Would you have bought that book just to get that one answer? Would you have known the book even had the information you needed without Google's help?
Post reply on HN