EasyList is in trouble and so are many ad blockers • Hacker News Board

Re: EasyList is in trouble and so are many ad blockers

#61

Post by tomschwiha » Wed, Oct 19, 2022, 7:44 PM UTC

tyingq wrote: post #33

tomschwiha wrote: post #15

I'm confused about the ToS comment by Cloudflare. The txt is on a website so it is a web content? So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.

It's from this tos page: https://www.cloudflare.com/terms/ 2.8 Limitation on Serving Non-HTML Content ...Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately... A huge text/plain artifact, requested often, would seem to fall into that category of "disproportionate percentage" compared to text/html served.

Cloudflare can decide whom they want to do business with. But a plain text file is in my opinion sort of HTML. At least it is not "non-html" content. A .pdf file would be non-HTML content.

What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS.

Re: EasyList is in trouble and so are many ad blockers

#62

Post by tyingq » Wed, Oct 19, 2022, 7:44 PM UTC

bornfreddy wrote: post #42

tyingq wrote: post #33

Earlier quoted context omitted.

It's from this tos page: https://www.cloudflare.com/terms/ 2.8 Limitation on Serving Non-HTML Content ...Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately... A huge text/plain artifact, requested often, would seem to fall into that category of "disproportionate percentage" compared to text/html served.

Sounds like it is meant to deal with multimedia mostly? But anyway, just rename .txt to .html and you're done.

I imagine that might help with automated tos rate limiting, but eventually someone at Cloudflare will probably cut them off. It's plain text, but it's basically serving a distributed database. And a hint at their scale is "100TB of “Access Denied” served up monthly.

Cloudflare just seems to be trying to limit the free tier to "caching website html for the purpose of showing it to humans". They have pricing and plans for things other than that.

Re: EasyList is in trouble and so are many ad blockers

#63

Post by sneak » Wed, Oct 19, 2022, 7:44 PM UTC

ignoramous wrote: post #34

MuffinFlavored wrote: post #12

> Even so, we continue to serve about 100TB of “Access Denied” pages monthly! What's the carbon footprint of this?

The thing with Access Denied is that these deprived clients retry with some vengeance. So, you're instead draining more resources than you'd like. I run a content-blocking DoH resolver, and this happened to us when we blocked IPs in a particular range and the result was... well... a lot of bandwidth for nothing.

Why serve any HTTP replies to those at all? If you are doing it at the IP level, why not just drop all inbound packets from the L3 address?

Re: EasyList is in trouble and so are many ad blockers

#64

Post by fweimer » Wed, Oct 19, 2022, 7:44 PM UTC

pool.ntp.org hands out specific subdomains for large-scale pool users. This way, it is possible to retire service for a subset of users that use devices that aren't updated anymore and are misbehaving.

The traffic issue is not just punted to DNS service. It's possible to return a cachable 127.0.0.1 response, and it's somewhat rare for DNS caches to be constantly powered up and down and reach out directly to authoritative DNS servers.

Re: EasyList is in trouble and so are many ad blockers

#65

Post by fabianhjr » Wed, Oct 19, 2022, 7:44 PM UTC

JoyfulPanda wrote: post #32

I'm not sure, but is IPFS capable of solving the issue?

The gateways would similarly not be thrilled about it.

I would add a redirect to the makers of the browsers in question (so that the leechers got to deal with the traffic themselves; https://en.wikipedia.org/wiki/Inline_linking)

Re: EasyList is in trouble and so are many ad blockers

#66

Post by Raed667 » Wed, Oct 19, 2022, 7:44 PM UTC

I worked on an ad-blocker a few months ago. I made the decision to have the filter-list files hosted on our own domain and CDN (similar to what Adguard does with their filters.adtidy.org).

This was done for 2 reasons:

1- Avoid scenarios like this where you ship code (extension in this case) that is hard to update. Then make that code depend on external resources outside of your control.

2- Leak our users' IP addresses to each random hosting provider.

So the solution was simple: Run a CRON once a day then host the files ourselves. Pretty happy with that decision now.

Re: EasyList is in trouble and so are many ad blockers

#67

Post by stereo » Wed, Oct 19, 2022, 7:44 PM UTC

bionade24 wrote: post #54

nnopepe wrote: post #20

serve a modified version to rate limited IP's that only contains popular indian sites and I'm sure it'll be resolved in a day or two

Limit this to the specific headers of these Webbrowsers though, please.

They already have that part figured out. From the article:

> When we encountered a similar problem last year, we found a simple solution: block the undesired traffic from these apps. Even so, we continue to serve about 100TB of “Access Denied” pages monthly!

Re: EasyList is in trouble and so are many ad blockers

#68

Post by celsoazevedo » Wed, Oct 19, 2022, 7:45 PM UTC

Raed667 wrote: post #45

hnaccy wrote: post #18

can they not block Indian IPs from cloudflare dashboard

That would prevent 1.4 billion people from ad-blocking. Not sure if we want to use these kind of blanket measures as a first response.

The logic here is that it's better to block 1 country and keep it working for everyone else than to leave it as it is and break it for everyone.

It's not ideal, but until the problem is fixed/better solutions are found, I think it's a good "first response".

Re: EasyList is in trouble and so are many ad blockers

#69

Post by sneak » Wed, Oct 19, 2022, 7:45 PM UTC

Raed667 wrote: post #45

hnaccy wrote: post #18

can they not block Indian IPs from cloudflare dashboard

That would prevent 1.4 billion people from ad-blocking. Not sure if we want to use these kind of blanket measures as a first response.

No, it would block 1.4 billion people from that one specific URL.

Re: EasyList is in trouble and so are many ad blockers

#70

Post by eis » Wed, Oct 19, 2022, 7:46 PM UTC

Regarding the 100TB of Access Denied pages: just drop the connection instead.

To make the system more scalable: instead of directly serving the file, serve a bunch of URLs to mirrors plus a checksum. The client must pick one of them. You can randomize the URLs and maybe add some geo logic to it. Let people provide mirrors. An additional indiraction step like this can prove incredibly powerful for systems that need to scale massively.