41–50 of 243 posts
EDIT: The "paste a headline into Google" trick still works for me, though. If this continues to be the case, they will keep indexing, of course.
Great idea here guys
I just tried clicking on "Harper Lee, Author of ‘To Kill a Mockingbird,’ Dies at Age 89" from wsj.com's homepage and got the paywall. I then pasted the headline into google and clicked on it from Google results and did not get hit by the paywall.
they're probably doing some sort of a/b testing by selectively letting some clicks through
Bypassing the paywall is more unethical that blocking ads. It is one thing to have control over your own browser but another to steal something from another site. Also, isn't it illegal to bypass computer security?
Their server can choose to do what it wants with your request and you can choose what to do with the response it sends.
Are User-Agent headers legally protected identities?
And congratulations, you have likely just "exceeded authorized access" and committed a felony violation of the CFAA punishable by a fine or imprisonment for not more than 5 years under 18 U.S.C. § 1030(c)(2)(B)(i). From the ABA: "Exceeds authorized access is defined in the Computer Fraud and Abuse Act (CFAA) to mean "to access a computer with authorization and to use such access to obtain or alter information in the…
If they're now blocking clicks from Google, doesn't that mean that they're cloaking and violating the Google's Webmaster Guidelines [1]? [1]: https://support.google.com/webmasters/answer/66355?hl=en
Google is not okay with cloaking, but they will whitelist publishers if the publisher specifically includes a parameter that declares if the site requires registration or subscription. This is done in the sitemap. https://support.google.com/news/publisher/answer/74288?hl=en
If this is true, what WSJ is doing is called "cloaking" and should cause it to get de-indexed: https://support.google.com/webmasters/answer/66355?hl=en
conversely, everyone should actively cloak and use random generated numbers to dynamically serve variants of their content similar to how mapmakers use trap streets. That way, like, another company wouldn't be profiting directly off of their work and threatening to sort of, destroy their entire business if you disagreed.
Earlier quoted context omitted.
A App Engine wouldn't have a IP with a reverse DNS *.googlebot.com, would it?
Nope, but it does resolve to something .google and if you check netblocks.google.com it will appear there so they might not be limiting it to googlebot only at this point. https://support.google.com/a/answer/60764?hl=en
Am I alone in feeling like this is akin to a tutorial on how you can shoplift without getting caught? WSJ, for better or worse, does not want to give you content without your paying for it. If you take that content without paying, you are stealing. Just because you have figured out how to get past their security does not mean it's not stealing. (See the second precept here: https://en.wikipedia.org/wiki/Five_Precepts…
Am I alone in feeling like this is akin to a tutorial on how you can shoplift without getting caught? WSJ, for better or worse, does not want to give you content without your paying for it. If you take that content without paying, you are stealing. Just because you have figured out how to get past their security does not mean it's not stealing. (See the second precept here: https://en.wikipedia.org/wiki/Five_Precepts…