Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EasyList is in trouble and so are many ad blockers

adguard.com

Post reply on HN
Print view View original

81–90 of 439 posts

Re: EasyList is in trouble and so are many ad blockers

#81

Post by ignoramous »

tyingq wrote: post #33
tomschwiha wrote: post #15

I'm confused about the ToS comment by Cloudflare. The txt is on a website so it is a web content? So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.

It's from this tos page: https://www.cloudflare.com/terms/ 2.8 Limitation on Serving Non-HTML Content ...Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately... A huge text/plain artifact, requested often, would seem to fall into that category of "disproportionate percentage" compared to text/html served.

This limitation apparently doesn't apply to R2 / Workers [0].

May be EasyList could host them there? That's what we do [1] (and the dashboards show 400TB+ per mo [2], likely rigged by the traffic between Workers and Cloudflare Cache).

[0] https://news.ycombinator.com/item?id=20791660

[1] https://news.ycombinator.com/item?id=30034547

[2] https://nitter.net/rethinkdns/status/1546232186554417152

Re: EasyList is in trouble and so are many ad blockers

#82

Post by kenmacd »

tomschwiha wrote: post #15

I'm confused about the ToS comment by Cloudflare. The txt is on a website so it is a web content? So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.

Imagine you're trying to block a DDoS attack. If the client is downloading HTML then they likely also have JS enabled giving you a ton of options for running code on their computer to help you decide if the traffic is legitimate.

If they're downloading text you can still use the headers, and some tricks around redirects, but overall you have far less data on which to decide.

Re: EasyList is in trouble and so are many ad blockers

#83

Post by Joel_Mckay »

Rate-limit the GeoIP list for the affected areas to drop if more than 20% of active traffic. i.e. the service outages get co-located only with the problem users areas.

Also, when doing auto-updates: always add a chaotic delay offset 1 to 180 minutes to distribute the traffic loads. Even in an office with 16 hosts or more this is recommended practice to prevent cheap routers hitting limits. Another interesting trend, is magnet/torrent being used for cryptographic-signed commercial package file distribution.

Free API keys are sometimes a necessary evil... as sometimes service abuse is not accidental.

Re: EasyList is in trouble and so are many ad blockers

#84

Post by bluehatbrit »

fdfaz wrote: post #24
hnaccy wrote: post #18

can they not block Indian IPs from cloudflare dashboard

I think the free plan allows this. Seems an easy solution.

I'll have a huge impact on anything making legitimate use of the list. Adblockers on sensible browsers will stop working etc.

It may be easy, and it may even be the only option, but it's a bad one that will need some thought from the maintainers I expect.

Re: EasyList is in trouble and so are many ad blockers

#85

Post by ziml77 »

laundermaf wrote: post #55
rc_mob wrote: post #19

Phew. Is just a bandwidth issue. This goofy title made me think advertisers found a way around ad blockers.

This issue caused CF to irreversibly ban them though, so it's not "just a bandwidth issue" anymore. > Based on the URL that are being requested at Cloudflare, it violates our ToS as well. All the requests are txt file extension which isn't a web content > you cannot use Cloudflare to cache or proxy the request to these text files

Where did you get that they were irreversibly banned? Or banned at all for that matter?

Re: EasyList is in trouble and so are many ad blockers

#86

Post by DigitallyFidget »

ignoramous wrote: post #34
MuffinFlavored wrote: post #12

> Even so, we continue to serve about 100TB of “Access Denied” pages monthly! What's the carbon footprint of this?

The thing with Access Denied is that these deprived clients retry with some vengeance. So, you're instead draining more resources than you'd like. I run a content-blocking DoH resolver, and this happened to us when we blocked IPs in a particular range and the result was... well... a lot of bandwidth for nothing.

But why bother with deny? Just send a blank text file (or one with as minimal data as needed to satisfy the rogue adblock) to the "blocked IPs" to mitigate the traffic for now. If firewall access exists, just drop the offending incoming traffic entirely.

Re: EasyList is in trouble and so are many ad blockers

#87

Post by Slix »

tyingq wrote: post #33
tomschwiha wrote: post #15

I'm confused about the ToS comment by Cloudflare. The txt is on a website so it is a web content? So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.

It's from this tos page: https://www.cloudflare.com/terms/ 2.8 Limitation on Serving Non-HTML Content ...Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately... A huge text/plain artifact, requested often, would seem to fall into that category of "disproportionate percentage" compared to text/html served.

This doesn't sound right to me. Cloudflare also protects web APIs. This text file is an extremely simple web API, but it is still a web API.

Re: EasyList is in trouble and so are many ad blockers

#88

Post by bombcar »

stereo wrote: post #67
bionade24 wrote: post #54

Earlier quoted context omitted.

Limit this to the specific headers of these Webbrowsers though, please.

They already have that part figured out. From the article: > When we encountered a similar problem last year, we found a simple solution: block the undesired traffic from these apps. Even so, we continue to serve about 100TB of “Access Denied” pages monthly!

100TB of Access Denied is only 38 MB/s, so not even a minor DDOS these days.

Re: EasyList is in trouble and so are many ad blockers

#89

Post by ignoramous »

sneak wrote: post #63
ignoramous wrote: post #34

Earlier quoted context omitted.

The thing with Access Denied is that these deprived clients retry with some vengeance. So, you're instead draining more resources than you'd like. I run a content-blocking DoH resolver, and this happened to us when we blocked IPs in a particular range and the result was... well... a lot of bandwidth for nothing.

Why serve any HTTP replies to those at all? If you are doing it at the IP level, why not just drop all inbound packets from the L3 address?

We were on Netifly way back then. So, no L3 blocks. Now on pages.dev and workers.dev, but haven't needed to enforce any rules yet.

Re: EasyList is in trouble and so are many ad blockers

#90

Post by DigitallyFidget »

sneak wrote: post #63
ignoramous wrote: post #34

Earlier quoted context omitted.

The thing with Access Denied is that these deprived clients retry with some vengeance. So, you're instead draining more resources than you'd like. I run a content-blocking DoH resolver, and this happened to us when we blocked IPs in a particular range and the result was... well... a lot of bandwidth for nothing.

Why serve any HTTP replies to those at all? If you are doing it at the IP level, why not just drop all inbound packets from the L3 address?

This is what I was wondering. I'm taking a wild guess that maybe they don't have that level of firewall access and it was being done through filtering by the webserver to provide an access denied.
Post reply on HN