Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Getting around website paywalls with devtools alone

bbarrows.com

Post reply on HN
Print view View original

51–60 of 123 posts

Re: Getting around website paywalls with devtools alone

#51

Post by 1vuio0pswjnm7 »

Another option no one has menitoned is AMP. Many sites that try to use "paywalls" have AMP URLs which point to pages that have all the full text of the article in

tags. These AMP sites generally look great in a text-only browser that does not run Javascript. Popular example is WSJ. In the URL, add /amp before /article/.

Paywalls are insidious because they target non-subscribers. Why let non-subscribers view articles. Why not password protect all subscriber content. Paywalls are a way to make money from (the attention of) non-subscribers, targeting them with ads and tracking. The strategy is apparently to annoy people to the point of subscribing. Yet even if they subscribe they will still be subjected to advertising. One potential advantage is that a paying subscriber has an enforceable contract. In theory the contract could contain enforceable privacy protections. "Tech" companies would never agree to give people enforceable privacy protections; it would destroy their "business".

The way to save journalism, especially local news, is to regulate "Big Tech" middlemen, who generally do not employ journalists and produce zero content.1 The quality of journalism in general has taken a nosedive, but placing the blame for that on web users not purchasing subscriptions is conveniently ignoring the true culprit.

1. Arguably that's a prerequisite to maintaining their Section 230 protection. In the recent Supreme Court oral arguments, Google's counsel argued Google is not a publisher. Then minutes later she argued Google has to make design decisions "like any publisher", therefore Google gets a free pass to reorganise information in annoying and perhaps harmful ways to maximise ad services revenue, like inserting "popular" videos into YouTube search results that have nothing to do with the query string.

Re: Getting around website paywalls with devtools alone

#53

Post by karrotwaltz »

albert_e wrote: post #33
karrotwaltz wrote: post #13

I use this JS bookmarklet to remove fixed elements and restore scrolling, it works most of the time: https://pastebin.com/qBjJHkMv I also have one to kill all running javascript and remove all event listeners, it works wonders when you are redirected to a paywall / login page after a few seconds.

Would you mind sharing the second script as well? Thanks This is supposed to be saved as a Javascript Bookmarklet?

Javascript killer: https://pastebin.com/utE3275J

Yes, I'm using it as a bookmarklet. I'm using firefox but I think it should work the same for other browsers.

Re: Getting around website paywalls with devtools alone

#54

Post by dkdbejwi383 »

samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

> But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

While there are some paywalled websites that allow you to read _n_ articles per period for "free", there are many that don't. How do I know in this case whether it's worth the cost?

There are also times where I'll see a link to something behind a paywall with an interesting headline (frequently on HN), but from a publication I don't regularly read, so have no intention of a subscription. It would be nice in this case to be able to pay a one-time, small contribution.

Worth stating I don't disagree necessarily with the sentiment, there are just a few "edge cases" that make it impractical.

Re: Getting around website paywalls with devtools alone

#55

Post by donohoe »

mik1998 wrote: post #46
phemartin wrote: post #43

Earlier quoted context omitted.

Is this even legal? How can sites host other ones' content, and that's ok? - Is it fair use because it's "archiving" the web? - Is it because it's on the open web and it's public domain? - Or is it illegal, and people do it because they can ¯\_(ツ)_/¯

Is caching illegal?

What archive.ph does is not just caching, it is hosting.

Technically it is illegal. Do I use it? Yes.

Re: Getting around website paywalls with devtools alone

#56

Post by donohoe »

izietto wrote: post #35
samwillis wrote: post #11

Many sites don't contains the full content even if you do that. I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases. But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

> But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there I'd go further than your statement: I try not to read paywalled contents. Actually I don't get all these workarounds about paywalls. I'm like "they don't want me to read it? I'm not going to read it then".

But they do want you to read. They also want to be financially viable.

Re: Getting around website paywalls with devtools alone

#57

Post by donohoe »

1vuio0pswjnm7 wrote: post #51

Another option no one has menitoned is AMP. Many sites that try to use "paywalls" have AMP URLs which point to pages that have all the full text of the article in tags. These AMP sites generally look great in a text-only browser that does not run Javascript. Popular example is WSJ. In the URL, add /amp before /article/. Paywalls are insidious because they target non-subscribers. Why let non-subscribers view articles.…

But the number of AMP versions is dwindling as Google is no longer forcing it.

Re: Getting around website paywalls with devtools alone

#58

Post by izietto »

donohoe wrote: post #56
izietto wrote: post #35

Earlier quoted context omitted.

> But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there I'd go further than your statement: I try not to read paywalled contents. Actually I don't get all these workarounds about paywalls. I'm like "they don't want me to read it? I'm not going to read it then".

But they do want you to read. They also want to be financially viable.

Sure, they want me to read if I think they are worth to have my subscription. But I don't think they are worth to have my subscription, otherwise I would subscribe them. So I don't read.

Re: Getting around website paywalls with devtools alone

#59

Post by donohoe »

jwr wrote: post #31

What I find annoying about paywalled sites is that they provide the full content to Google. And Google is OK with indexing the full content, even though it is not available on the internet, and even though they explicitly forbid the practice of showing different content to a search engine from what is available publicly. Paywalled sites are just fine, but they are not part of the open Internet, and should not pretend…

> they explicitly forbid the > practice of showing different > content to a search engine from > what is available publicly

This isn’t true. This paywall treatment is something they do allow and have worked to accommodate.

Re: Getting around website paywalls with devtools alone

#60

Post by donohoe »

alkonaut wrote: post #30

10 years ago paywalled sites contained the content just hidden. Today I haven't seen a site in a long time that renders the content hidden (why would it do that? There is no reason to do it based on indexing/SEO as far as I'm aware). Even cached/archived versions these days tend to not include the whole text. Basically: they figured out how to make a paywall, which frankly isn't that surprising.

Not sure I agree with that assessment.

There are so many ways to do a paywall and you’ll still see all sorts of flavors across the web today.

Post reply on HN