Wait, Google Sent Me

#1

Post by jashkenas » Tue, May 12, 2015, 2:23 PM UTC

Wait, Google Sent Me
chrome.google.com

Re: Wait, Google Sent Me

#2

Post by jashkenas » Tue, May 12, 2015, 2:23 PM UTC

Source code: https://gist.github.com/jashkenas/9a4d9e648eb11aab394e

Re: Wait, Google Sent Me

#3

Post by dreen » Tue, May 12, 2015, 2:53 PM UTC

jashkenas wrote: post #2

Source code: https://gist.github.com/jashkenas/9a4d9e648eb11aab394e

Could you give more information on how do some websites react to a google referer?

Re: Wait, Google Sent Me

#4

Post by jebronie » Tue, May 12, 2015, 2:57 PM UTC

Google users are priviliged for a reason.

Re: Wait, Google Sent Me

#5

Post by patio11 » Tue, May 12, 2015, 2:58 PM UTC

dreen wrote: post #3

jashkenas wrote: post #2

Source code: https://gist.github.com/jashkenas/9a4d9e648eb11aab394e

Could you give more information on how do some websites react to a google referer?

The design intent is almost certainly using Google's "first click free" policy, which primarily applies to newspapers like the NYT or WSJ. Google back in the day struck a deal with the print media: they would remain in Google's index if and only if clicking from a Google search to their articles circumvented their paywall on the first click. If they put only partial content or a full paywall, they'd be delisted. It was not exactly a secret that this would mean that Google would rank scrapers higher for NYT articles than they'd rank the NYT.

Thus, when someone posts a paywalled link to HN, someone inevitably posts the advice "Google the title and click through." (Other than, say, "If you're a working professional, just buy a WSJ and NYT subscription.")

Re: Wait, Google Sent Me

#6

Post by lfuller » Tue, May 12, 2015, 2:58 PM UTC

dreen wrote: post #3

jashkenas wrote: post #2

Source code: https://gist.github.com/jashkenas/9a4d9e648eb11aab394e

Could you give more information on how do some websites react to a google referer?

Many news websites will pop up a paywall if you visit a link directly, but won't if you arrive from a Google search.

Re: Wait, Google Sent Me

#7

Post by [deleted] » Tue, May 12, 2015, 2:59 PM UTC

dreen wrote: post #3

jashkenas wrote: post #2

Source code: https://gist.github.com/jashkenas/9a4d9e648eb11aab394e

Could you give more information on how do some websites react to a google referer?

[deleted]

Re: Wait, Google Sent Me

#8

Post by jfoster » Tue, May 12, 2015, 2:59 PM UTC

It would be nice if Google could make some of the Chrome permissions a bit more fine-grained.

This one needs to "read and change your data on all websites", but with a better extensions architecture could be something much less scary like "read and change your data on the current website whenever you click the extension's icon".

Re: Wait, Google Sent Me

#9

Post by sp332 » Tue, May 12, 2015, 3:02 PM UTC

jebronie wrote: post #4

Google users are priviliged for a reason.

I thought Google users were only privileged because news sites want the full text of the page to show up in searches, and Google punishes pages that show different content to the web crawler than they show to searchers.

Re: Wait, Google Sent Me

#10

Post by cissou » Tue, May 12, 2015, 3:03 PM UTC

jebronie wrote: post #4

Google users are priviliged for a reason.

and the reason is: if a user bounces off a SEO result, the ranking is degraded, and nobody wants that, so Google users can skip the paywalls. I don't think that's a good reason, or even a good way to protect content.