I find it amazing how often ad blocking is discussed here, and start to wonder how many peeps hanging out here on the other hand depend indirectly on ad revenue to pay bills?
There are obviously the big corps Facebook and Google, but also my own small employer, which is in theory in a different biz, runs ads on the web shop as an additional income source (which I find not very clever, increases page load times and is simply not our core biz). Do we all pay our bills with money made from the pour souls who did not get around to install an ad blocker or advanced setups like a pi hole?
I'm on the pro ad blocking camp personally, also worked in the past in a biz that was 100% ad financed for a short period of time. Also there the whole tech department was using ad blockers.
Ad blocking with Raspberry Pi and Pi-hole
121–130 of 163 posts
Re: Ad blocking with Raspberry Pi and Pi-hole
#122I actually have been doing some work with MaraDNS to have the ability to have a pi-hole sized blacklist. The main source of pi-hole’s blacklist is this Git repo: https://github.com/StevenBlack/hosts That is about 60,000 hosts, so I added MaraDNS support to have up to 500,000 blacklisted names. Since it’s a speed-optimized (not size-optimized) cache, each element takes about a kilobyte of memory, so a blacklist this s…
> Finding ways to have, say, 10 million blacklist elements with a small memory footprint is left as an exercise for the reader.
In case the reader is interested (and this may not be what OP had in mind): In trading memory gains with computation, this is how I squeezed ~4M entries in less than 30 MiB:
1. Compress (reversed) individual entries using a dictionary or entropy encoder. I experimented with Shoco [0], Femto [1], and a modified LZ78 [2].
2. Create a bloom filter for membership checks, [3] or any probabilistic structure for that matter, like the xor-filters [4]. This prevents expensive but unnecessary lookups.
3. Insert the compressed (reversed: google.com -> com.google) entries into a radix-tree [5]. You could use a directed-acyclic graph (DAG), if you don't need to store key-value pairs.
4. Compact [6][7] the radix-tree [8] (compacting a DAG might be super complicated).
5. On every search:
5a. Compress (reversed) incoming query (i.e. domain-name).
5b. Check the bloom filter.
5c. Traverse the radix-tree, if necessary.
5d. Cache an eligible entry in a LFU [9] or possibly, using any other suitable algorithm inspired from Linux's page replacement implementations [10].
6. Strive for zero-allocation and zero-gc.
---
[0] https://news.ycombinator.com/item?id=10060018
[1] https://github.com/gtoubassi/femtozip/wiki/How-femtozip-work...
[2] https://en.wikipedia.org/wiki/LZ78
[3] https://news.ycombinator.com/item?id=12231623
[4] https://news.ycombinator.com/item?id=21840821
[5] https://vincent.bernat.ch/en/blog/2017-ipv4-route-lookup-lin...
[6] https://news.ycombinator.com/item?id=22544718
[7] https://www.youtube-nocookie.com/embed/3Y2weLDiUWw
[8] https://news.ycombinator.com/item?id=2348619
[9] https://github.com/ronomon/hash-table/blob/master/README.md
[10] https://en.wikipedia.org/wiki/Page_replacement_algorithm
Re: Ad blocking with Raspberry Pi and Pi-hole
#123Earlier quoted context omitted.
It's a tragic move. The DHCP/DNS ecosystem made managing devices dead simple. OTOH I suppose we were foolish to ever think our devices were playing nice. Is there a fix to DNS-over-HTTPS as a network operator? Can you MITM your "own" proprietary devices? What dragons live there?
You can set a canary domain which currently is used The only other way would be MITM all https traffic and sign with your own certificate, but many iot devices won’t allow you to install a new certificate store.
I don’t know what a canary domain is. That reddit post isn’t entirely clear but it seems to be some kind of DNS black hole. That would rely on the application using your config.
Will a canary domain on an edge device prevent my smart TV from using DoH to get the IP of a tracking server?
Re: Ad blocking with Raspberry Pi and Pi-hole
#124Pi-hole has been excellent. I was able to discover that my Samsung TV was reporting minute by minute updates on what I was watching to a local Australian company. Unplugged it faster than I could swear.
I'm sure you know this, but with a reasonable router you can redirect non pihole port 53 traffic back to the pihole. A relatively cheap Edgerouter lite does this for me.
Re: Ad blocking with Raspberry Pi and Pi-hole
#125I find it amazing how often ad blocking is discussed here, and start to wonder how many peeps hanging out here on the other hand depend indirectly on ad revenue to pay bills? There are obviously the big corps Facebook and Google, but also my own small employer, which is in theory in a different biz, runs ads on the web shop as an additional income source (which I find not very clever, increases page load times and is…
I can definitely agree with that. You can always explicitly whitelist a site you would like to see ads on and thus support, like I do
Re: Ad blocking with Raspberry Pi and Pi-hole
#126Earlier quoted context omitted.
I've set it up for most of my close family and friends, added a physical button to the top of the Raspberry Pi case that disables it for x minutes (x changes depending who they are and their needs), so if they're having issues they go press the button to access the problematic website. I keep reading about people having to disable the Pi-Hole so much that it becomes annoying, or constantly butting heads with websites…
> What websites are people visiting that the Pi-Hole doesn't work with? Usually sites where the dev has based functionality on a JS module loaded from an advertise/tracker site, and that is being blocked, resulting in missing functionality. Two prominent examples that I've personally had to deal with: CVS, and Taco Bell's iOS app. Edit: Oh, and google's inserted redirects in shopping results lists are blocked by defa…
Airline metasearch engines like Skyscanner & Kayak use all kinds of advertisement and tracking trickery.
Re: Ad blocking with Raspberry Pi and Pi-hole
#127some note for those want to use pihole - in windows, if you use primary dns as pihole and secondary dns as another cloud option (cloudflare, google), some ads will go through. secondary dns is not failover dns. try it and you will see.
I don't think this is just Windows. I read the pihole documentation and came to the conclusion that primary/secondary DNS was more "a dns server" and "another dns server".
Re: Ad blocking with Raspberry Pi and Pi-hole
#128Earlier quoted context omitted.
Will you test the efficiency of the block list. For example, given a block list of 60,000 how many hosts on the list did your computer actually try to access. Have you ever read through one of those massive blocklists. I use a whitelist rather than a blocklist. Similar to a firewall, I block everything by default. Then carefully choose what I allow. This way I can see exactly what hosts I actually need to access. Mak…
MaraDNS (OK, Deadwood) can handle white lists too: upstream_servers = {} upstream_servers["."] = "192.168.253.253" # Never answers upstream_servers["good-domain.example.com."] = "9.9.9.9" upstream_servers["whitelist-entry.foo."] = "9.9.9.9" # And so on The downside is that the code currently only supports 20,000 elements added this way.
I am a djbdns user. I also use nsd and unbound. I do not use "upstream" third party DNS, such as 9.9.9.9. Surprised to hear that anyone still uses MaraDNS.
What I am curious about are these massive blocklists. Does anyone actually read through them. Does it make sense to block 60,000 hosts when only, say 100 or even 1000, ever stand a chance of being accessed by a user's computer -- due to the user's particular usage habits.
Re: Ad blocking with Raspberry Pi and Pi-hole
#129Earlier quoted context omitted.
Instead of building hash tables with buckets (and a linked list in case of hash collision), why not use a flat array based hash table and open addressing?
I actually tried that with MaraDNS in the pre-1.0 days (in case of collision, go forward in hash array N elements until there is no collision). The problem is that it works really great for static data, but doesn’t work for dynamic data as cleanly as using a linked list to handle collisions. The one thing I would do differently is that Deadwood (MaraDNS’s recursive/blacklist DNS server) adds individual elements with…
But the data is fairly static isn't it? You're not constantly updating the blacklists?
Re: Ad blocking with Raspberry Pi and Pi-hole
#130Earlier quoted context omitted.
MaraDNS (OK, Deadwood) can handle white lists too: upstream_servers = {} upstream_servers["."] = "192.168.253.253" # Never answers upstream_servers["good-domain.example.com."] = "9.9.9.9" upstream_servers["whitelist-entry.foo."] = "9.9.9.9" # And so on The downside is that the code currently only supports 20,000 elements added this way.
I am a djbdns user. I also use nsd and unbound. I do not use "upstream" third party DNS, such as 9.9.9.9. Surprised to hear that anyone still uses MaraDNS. What I am curious about are these massive blocklists. Does anyone actually read through them. Does it make sense to block 60,000 hosts when only, say 100 or even 1000, ever stand a chance of being accessed by a user's computer -- due to the user's particular usage…
Well, why not? 60.000 isn't a huge amount for a modern computer. If they're known to contain ads/trackers you want to block, then why not include them?
It might make sense though to do a periodic run through them to see if they still exist.