Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EasyList is in trouble and so are many ad blockers

adguard.com

Post reply on HN
Print view View original

381–390 of 439 posts

Re: EasyList is in trouble and so are many ad blockers

#381

Post by Programmatic »

Joel_Mckay wrote: post #190
codalan wrote: post #135

Earlier quoted context omitted.

That would only work if they had an API; AFAICT, they're just hosting a file. At this point, they might be better off coordinating with the other major adblocker providers and just outright move the file elsewhere. Breaking other people's garbage code is better than breaking yourself trying to fix it. Especially on a budget of $0.00. If the defective code for the browsers are in public repos, it might also be more ef…

It is very common to see API keys in urls for access to what are essentially flat files. Thus, fairly trivial to change from: https://127.0.0.1/file.csv to https://127.0.0.1/file.csv?apikey=abc123 This could allow client specific quotas, and easy adoption with maintained projects in minutes. Thus, defective and out-of-maintenance projects would need manually updated or get a 404. =)

API keys are most successful when they're issued for server-side use; when used client-side the usual pattern that I see is for individual clients to request their own API key?

In this case, it would need to be distributed to myriad users who legitimately need to ask for the lists and then could be scraped by the "attacker", but at least then they'd have to be knowingly malicious vs. accidentally malicious.

Re: EasyList is in trouble and so are many ad blockers

#382

Post by kevincox »

thiht wrote: post #372
squarefoot wrote: post #364

This seems the perfect use case for letting a secure BitTorrent tracker share the lists, then either implementing the client in the browser, or having it as a system service that syncs the necessary files.

Torrent was also my first thought. Is there any reason why this is not a good solution?

One downside is that Torrent doesn't really share blocks between torrents. So updating the torrent would result in a full download.

That being said the data is small so it probably isn't a big deal.

A better solution may be something like IPFS where with a rolling-checksum chunking algorithm you only need to download the changed parts of the list. You could even use IPNS to distribute updates for a fully decentralized setup.

Re: EasyList is in trouble and so are many ad blockers

#383

Post by kevincox »

Programmatic wrote: post #381
Joel_Mckay wrote: post #190

Earlier quoted context omitted.

It is very common to see API keys in urls for access to what are essentially flat files. Thus, fairly trivial to change from: https://127.0.0.1/file.csv to https://127.0.0.1/file.csv?apikey=abc123 This could allow client specific quotas, and easy adoption with maintained projects in minutes. Thus, defective and out-of-maintenance projects would need manually updated or get a 404. =)

API keys are most successful when they're issued for server-side use; when used client-side the usual pattern that I see is for individual clients to request their own API key? In this case, it would need to be distributed to myriad users who legitimately need to ask for the lists and then could be scraped by the "attacker", but at least then they'd have to be knowingly malicious vs. accidentally malicious.

You generally add a small "cost" to request an API key. For example submit your email to this form and wait a day.

Then browser makes like this will not reasonable be able to request a new key automatically for every install. So they will just request one and ship it.

Then when you get abuse like this you can disable it.

Re: EasyList is in trouble and so are many ad blockers

#384

Post by greyhair »

So I understand some of the comments streaming down on CloudFlare, but I would like to get to another point entirely.

So people writing apps using crappy coding are cratering Easylist basically through unintentional DDoS. It is the apps that suck. Full Stop.

I have noted over the last five to ten years the general retrograde usability of phone apps and full desktop apps in general. Bad UIs, inconsistent behavior, performance issues on stuff that, at least on the surface, appears to be trivial.

I don't understand what is causing the general slide in quality, but it is clearly visible, and it seems to me, untenable over the near term.

Is it the tools? Is it the app churn pressure? I really do not know, because I work in a very different part of the industry. We have our own issues there, but that has more to do with technology churn (particularly in wireless standards) than in the tools and platforms.

So what is up with the app world, in general, and the web app world in particular? I am all ears, because as I said, I don't work in that space.

Re: EasyList is in trouble and so are many ad blockers

#385

Post by kevincox »

srvmshr wrote: post #308
ignoramous wrote: post #216

Earlier quoted context omitted.

> Bittorrent, switch in long-term to that. S3 buckets in IAD with I'd imagine, some tech IPFS/Filecoin/Sia might come in handy, too, but unsure of how healthy most of these web3 projects are right now. There's also fosstorrents.com that help seed projects.

Is IPFS strictly Web3? Sure it gained traction around Blockchain, crypto DeFI - but the storage technology is ELI5 a massive distributed storage. I could hazard a guess that in terms of philosophy its closer to BitTorrent than Blockchain

IPFS is just distributed content discovery and download.

Someone advertises the content to the network, then people looking for the content can find it. Usually the people who find it advertise that they now have it to the network too.

It has nothing to do with cryptocurrency but is commonly used as a great way to embed "larger" content into blockchains and other immutable stores. It works well for this because 1. The CID contains a cryptographic hash of the expected content 2. You can change where/how you store the actual content without updating the URL.

Re: EasyList is in trouble and so are many ad blockers

#386

Post by kevincox »

muststopmyths wrote: post #13
disadvantage wrote: post #2

> There’s an open source Android browser (now seemingly abandoned) that implements ad-blocking functionality > The problem is that this browser has a very serious flaw. It tries to download filters updates on every startup, and on Android it may happen lots of times per day. It can even happen when the browser is running in the background EasyList should be offered as a version-controlled copy you grab once, that the…

Call me lazy but I’d just support an If-modified-since header in such a simple case and call it good

The list does support an ETag and the 304 even appears to not be rate-limited by Cloudflare.

Re: EasyList is in trouble and so are many ad blockers

#387

Post by kevincox »

self_awareness wrote: post #343

Stupid question, but can't they rename the txt to .html, put on the front, on the back, and use cloudflare to cache/proxy the content?

I suspect that the real problem isn't the type of content but the fact that it is being requested by robots at a high rate. If you had a text file that humans were actually browsing and reading I doubt Cloudflare would care, but humans read files at a much lower rate than is occurring for this text file.

Re: EasyList is in trouble and so are many ad blockers

#388

Post by bombcar »

aembleton wrote: post #329
bombcar wrote: post #295

Just serve an easy list to these browsers that blocks every image and all css. Problem solved.

How do you determine that it is these browsers requesting it and not uBO? It would be easy for the browsers to set their useragent to something like Chrome.

The browsers don’t update - they’re already returning 100b access denied or something.

Re: EasyList is in trouble and so are many ad blockers

#389

Post by arglebargle123 »

mensetmanusman wrote: post #177

Perfect application for a functioning system like torrent.

You'd still need the coordinator node (tracker) for every single request, so you'd still be serving an absolutely insane amount of traffic off one central box.

If you baked something like IPFS or DHT torrent capability into the application and requested your blocklists that way you'd solve the single point of failure problem, but that's asking a whole lot from a shittily maintained and poorly configured browser fork.

Re: EasyList is in trouble and so are many ad blockers

#390

Post by arglebargle123 »

ignoramous wrote: post #34
MuffinFlavored wrote: post #12

> Even so, we continue to serve about 100TB of “Access Denied” pages monthly! What's the carbon footprint of this?

The thing with Access Denied is that these deprived clients retry with some vengeance. So, you're instead draining more resources than you'd like. I run a content-blocking DoH resolver, and this happened to us when we blocked IPs in a particular range and the result was... well... a lot of bandwidth for nothing.

That reminds me of the absolutely insane amount of traffic my mother's Roku TV shits out when it can't resolve/reach its spyware and telemetry services. It's like 95-98% of the blocked traffic on her network.

Is there a clean solution to this problem these days? Like some kind of adblocking router that resolves these addresses correctly but then routes packets destined for these services into a black hole so the requests eventually timeout? That would at least slow the repeat request floods down significantly.

Post reply on HN