Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Getting around website paywalls with devtools alone

bbarrows.com

Post reply on HN
Print view View original

81–90 of 123 posts

Re: Getting around website paywalls with devtools alone

#81

Post by jrochkind1 »

phemartin wrote: post #43
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

Is this even legal? How can sites host other ones' content, and that's ok? - Is it fair use because it's "archiving" the web? - Is it because it's on the open web and it's public domain? - Or is it illegal, and people do it because they can ¯\_(ツ)_/¯

Is Firefox "reader" mode legal?

Is Google cache legal?

What about Internet Archive wayback machine archive?

Is deleting your cookies legal? How about spoofing your user-agent?

How about a browser plugin that automates what OP describes?

In the case of caches like Google, Internet Archive, or `archive.today` (same thing as `archive.ph`)... probably, in the USA? If it winds up in court, we will find out, eventually.

Simply reading anything on the web technically involves "making a copy" already, which is one reason it gets and has remained somewhat confusing and complex to determine what is or is not legal with regard to copying web content. You can't simply say "making a copy is not allowed".

Re: Getting around website paywalls with devtools alone

#82

Post by navigate8310 »

mik1998 wrote: post #79
donohoe wrote: post #55

Earlier quoted context omitted.

What archive.ph does is not just caching, it is hosting. Technically it is illegal. Do I use it? Yes.

How do you "cache" without "hosting"? What is the technical difference between those two acts?

Caching is parroting what the original poster publishes. If the OP asks not to index, you cease indexing. However, if you are purposeful "hosting," you may not accept and continue hosting, knowing all the legal consequences that may follow.

Re: Getting around website paywalls with devtools alone

#83

Post by enumjorge »

samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I’d be more willing to subscribe if publications didn’t pull the dark pattern of making signing up fast and easy, but then requiring a call to a rep to cancel.

I used to subscribe to the nytimes but a few years ago I needed a break from news. My plan was to come back in 6-12 months, but they made me wait on the phone for 25 mins for something that should have taken a couple of minutes on their site. I cancelled and never went back.

Re: Getting around website paywalls with devtools alone

#84

Post by sowbug »

DennisP wrote: post #61
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I subscribe to a major newspaper. But I'm not going to subscribe to all major newspapers. The individual subscription model doesn't really fit a world where you can go to one site, and click links to articles from lots of different publications. If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read, I'd subscribe to that. Since they don't, I sub…

Syndication used to be the way this idea worked. You'd buy your local paper that wrote about the cat stuck in the tree down the street, but it also republished national articles from AP. If you really cared about another locale, you might subscribe to two papers, which meant you might see the same national article twice.

Re: Getting around website paywalls with devtools alone

#85

Post by mik1998 »

navigate8310 wrote: post #82
mik1998 wrote: post #79

Earlier quoted context omitted.

How do you "cache" without "hosting"? What is the technical difference between those two acts?

Caching is parroting what the original poster publishes. If the OP asks not to index, you cease indexing. However, if you are purposeful "hosting," you may not accept and continue hosting, knowing all the legal consequences that may follow.

Caching has nothing to do with indexing. If I set up an HTTP cache to serve cached websites on my network, it would be difficult for anyone to request to not cache the HTTP responses considering it is most likely they would not be aware of them at all. It is, however, exactly equivalent to what websites such as archive.ph are doing.

Re: Getting around website paywalls with devtools alone

#87

Post by Kerrick »

denton-scratch wrote: post #39
PufPufPuf wrote: post #16

I recommend the browser extension "Bypass Paywalls Clean". I sometimes think about the morality of using it, but I just don't find it viable to pay all the websites where I read just a single article.

> but I just don't find it viable to pay all the websites where I read just a single article. This. In the print days, you'd buy a newspaper; you'd have access to all the articles in that edition. I used to read a daily paper. In the modern world, these papers expect you to pay for a newspaper just to read a single article. I dunno, perhaps they could form a "Paywall Consortium", so that I could pay a one-day fee to…

> But if you want me to buy the whole paper to read a single article, well, ain't gonna happen.

This was common for non-subscribers in the print days. Newspapers would print a number of enticing headlines and images on the front page above the fold, and display those folded newspapers for sale at dispensers, newsstands, and stores. Many people who bought a one-off paper would buy for a single article that interested them.

Re: Getting around website paywalls with devtools alone

#88

Post by monksy »

enumjorge wrote: post #83
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I’d be more willing to subscribe if publications didn’t pull the dark pattern of making signing up fast and easy, but then requiring a call to a rep to cancel. I used to subscribe to the nytimes but a few years ago I needed a break from news. My plan was to come back in 6-12 months, but they made me wait on the phone for 25 mins for something that should have taken a couple of minutes on their site. I cancelled and n…

I was going to subscribe to NYTdigital .. but I saw the reviews talking about this experience.

Re: Getting around website paywalls with devtools alone

#89

Post by iicc »

jwr wrote: post #31

What I find annoying about paywalled sites is that they provide the full content to Google. And Google is OK with indexing the full content, even though it is not available on the internet, and even though they explicitly forbid the practice of showing different content to a search engine from what is available publicly. Paywalled sites are just fine, but they are not part of the open Internet, and should not pretend…

Sometimes the full content is available - but only if you navigate to it via a Google results page, and you don't have existing cookies implementing a "free article limit".

Re: Getting around website paywalls with devtools alone

#90

Post by every »

1vuio0pswjnm7 wrote: post #67
donohoe wrote: post #57

Earlier quoted context omitted.

But the number of AMP versions is dwindling as Google is no longer forcing it.

Yep. And the browser vendor can make changes to DevTools, archive.ph is already privacy-unfriendly and blocked in some countries, it could disappear, 12ft.io could disappear, and so on. The fact that seems least likely to change is that most "paywalls" rely on Javascript, CSS or some other "feature" of so-called "modern" web browsers. I have lost count of how many times an HN comment alleges "paywalled" in a thread a…

My lynx is always at the ready for paywalls and other annoyances. My guess is that showing up using such an archaic tool implies you might also be using a screen reader setting like blynx [1]. The more thoughtful sites will even give you a link to jump directly to content from the top...

[1] http://leb.net/blinux/blynx/

Post reply on HN