Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Getting around website paywalls with devtools alone

bbarrows.com

Post reply on HN
Print view View original

61–70 of 123 posts

Re: Getting around website paywalls with devtools alone

#61

Post by DennisP »

samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I subscribe to a major newspaper. But I'm not going to subscribe to all major newspapers. The individual subscription model doesn't really fit a world where you can go to one site, and click links to articles from lots of different publications.

If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read, I'd subscribe to that. Since they don't, I subscribe to one paper and do workarounds on the others. I feel this is ethical because if everyone did it, with a decently random distribution, then the newspapers would survive just fine. They'd make the same overall revenue as when everyone had one newspaper, showing up at their doorstep each morning.

Re: Getting around website paywalls with devtools alone

#63

Post by prettyStandard »

drewtato wrote: post #5

Usually on these sites, there'll be an `overflow: hidden` element that's holding all the content. If you can find and disable that CSS line, it'll work as normal. Or just save it to the Wayback Machine and read it through that.

Not sure if this is exactly right, but something like this should obviate the need to find the element holding all the content.

* { overflow: visible !important; }

Re: Getting around website paywalls with devtools alone

#64

Post by qingdao99 »

vie00001 wrote: post #15
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

> I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. I think for search engine crawlers there are versions without a paywall so these articles can get fully indexed. Archive.ph, and similar services, might get the full content this way somehow. But I am just guessing.

Yes, it's called dynamic rendering when you serve different content based on the user agent, but it's apparently not recommended anymore.

https://searchengineland.com/google-no-longer-recommends-usi...

Re: Getting around website paywalls with devtools alone

#65

Post by dspillett »

> But I didn't know how to scroll down on the page until today.

That is usually due to an "overflow: hidden" somewhere near the top of the DOM tree. Remove that and your normal scrollbar usually returns. You see this a lot with "accept being stalked to read this" pop-overs as well as paywall related shenanigans.

I've seen some sites put it back in via JS. There is probably a workaround for that too though the easiest one is to not worry about it and DNS blacklist the site so you don't waste time visiting it again in future.

Re: Getting around website paywalls with devtools alone

#66

Post by [deleted] »

DennisP wrote: post #61
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I subscribe to a major newspaper. But I'm not going to subscribe to all major newspapers. The individual subscription model doesn't really fit a world where you can go to one site, and click links to articles from lots of different publications. If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read, I'd subscribe to that. Since they don't, I sub…

[deleted]

Re: Getting around website paywalls with devtools alone

#67

Post by 1vuio0pswjnm7 »

donohoe wrote: post #57
1vuio0pswjnm7 wrote: post #51

Another option no one has menitoned is AMP. Many sites that try to use "paywalls" have AMP URLs which point to pages that have all the full text of the article in tags. These AMP sites generally look great in a text-only browser that does not run Javascript. Popular example is WSJ. In the URL, add /amp before /article/. Paywalls are insidious because they target non-subscribers. Why let non-subscribers view articles.…

But the number of AMP versions is dwindling as Google is no longer forcing it.

Yep. And the browser vendor can make changes to DevTools, archive.ph is already privacy-unfriendly and blocked in some countries, it could disappear, 12ft.io could disappear, and so on.

The fact that seems least likely to change is that most "paywalls" rely on Javascript, CSS or some other "feature" of so-called "modern" web browsers.

I have lost count of how many times an HN comment alleges "paywalled" in a thread and I am reading the article just like any other because I am not using a graphical web browser. I would not even know there was a "paywall". Paywalls have dependencies.

Re: Getting around website paywalls with devtools alone

#68

Post by k_sze »

DennisP wrote: post #61
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I subscribe to a major newspaper. But I'm not going to subscribe to all major newspapers. The individual subscription model doesn't really fit a world where you can go to one site, and click links to articles from lots of different publications. If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read, I'd subscribe to that. Since they don't, I sub…

How about getting a paid membership for certain higher education institutions’ libraries, which will give you access to not only books, but also a plethora of periodicals. And that will be fully legal and ethical.

Re: Getting around website paywalls with devtools alone

#69

Post by samwillis »

DennisP wrote: post #61
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

I subscribe to a major newspaper. But I'm not going to subscribe to all major newspapers. The individual subscription model doesn't really fit a world where you can go to one site, and click links to articles from lots of different publications. If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read, I'd subscribe to that. Since they don't, I sub…

> If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read

So some extent that's what Apple News+ (included in Apple One) is. But it doesn't do multi region stuff I don't think, and misses some major publications.

Re: Getting around website paywalls with devtools alone

#70

Post by tyingq »

donohoe wrote: post #59
jwr wrote: post #31

What I find annoying about paywalled sites is that they provide the full content to Google. And Google is OK with indexing the full content, even though it is not available on the internet, and even though they explicitly forbid the practice of showing different content to a search engine from what is available publicly. Paywalled sites are just fine, but they are not part of the open Internet, and should not pretend…

> they explicitly forbid the > practice of showing different > content to a search engine from > what is available publicly This isn’t true. This paywall treatment is something they do allow and have worked to accommodate.

It's also true that they say "cloaking" isn't allowed, "any type of cloaking"

https://youtu.be/QHtnfOgp65Q

Post reply on HN