Blocking system test

[WST16]

If you're implementing AdGuard or Pi-Hole, here's a page that checks how efficient your blocking system is.
You can manually amend what's missing to your system.
I got 87% for my AdGuard. Very nice but I'll try to improve it.

Ad Blocker Test

d3ward.github.io

 

[Saqr]

So you do install the apps and configure them and later audit their blocking performance ?
Are these two containerized in your docker instance or standalone apps ?

 

[WST16]

So you do install the apps and configure them and later audit their blocking performance ?

I stumbled on this page, so that’s the idea here at least.
To compare, I manually changed my DNS IP address to point to the ISP’s on the router (like any ordinary home), and I got a shocking 12% with almost everything in red.

Are these two containerized in your docker instance or standalone apps ?

Yes. But why two? Just one. AdGuard is running on Docker and I configured my router’s DHCP service to pass the AdGuard IP address as the DNS service to any client on the LAN. So clients will send their DNS requests to AdGuard on the Synology to resolve them. Another nice thing is that you can configure AdGuard to use DNS over https so even the DNS requests resolved by AdGuard are encrypted and your ISP cannot pry on you.

Here’s an installation video (referenced initially by @Telos) and I found it to be the best. Just mind the error corrected in the video description.

 

[Telos]

AdguardHome result:

Now... to round up those strays.

 

[WST16]

The list from the website ready to be copied to AdGuard

# Google Ads
||pagead2.googlesyndication.com^
||pagead2.googleadservices.com^
||ads.google.com^
||adservice.google.com^
||googleadservices.com^
#Media.net^
||static.media.net^
||media.net^
||adservetx.media.net^
#Doubleclick.net^
||mediavisor.doubleclick.net^
||m.doubleclick.net^
||static.doubleclick.net^
||doubleclick.net^
||ad.doubleclick.net^
#FastClick
||fastclick.com^
||fastclick.net^
||media.fastclick.net^
||cdn.fastclick.net^
#Amazon
||adtago.s3.amazonaws.com^
||analyticsengine.s3.amazonaws.com^
||advice-ads.s3.amazonaws.com^
||affiliationjs.s3.amazonaws.com^
||advertising-api-eu.amazon.com^
||amazonaax.com^
||amazonclix.com^
||assoc-amazon.com^
## Analytics ====================
# Yahoo
||ads.yahoo.com^
||adserver.yahoo.com^
||global.adserver.yahoo.com^
||us.adserver.yahoo.com^
||adspecs.yahoo.com^
||br.adspecs.yahoo.com^
||latam.adspecs.yahoo.com^
||ush.adspecs.yahoo.com^
||advertising.yahoo.com^
||de.advertising.yahoo.com^
||es.advertising.yahoo.com^
||fr.advertising.yahoo.com^
||in.advertising.yahoo.com^
||it.advertising.yahoo.com^
||sea.advertising.yahoo.com^
||uk.advertising.yahoo.com^
||analytics.yahoo.com^
||cms.analytics.yahoo.com^
||opus.analytics.yahoo.com^
||sp.analytics.yahoo.com^
||comet.yahoo.com^
||log.fc.yahoo.com^
||ganon.yahoo.com^
||gemini.yahoo.com^
||beap.gemini.yahoo.com^
||geo.yahoo.com^
||marketingsolutions.yahoo.com^
||pclick.yahoo.com^
||analytics.query.yahoo.com^
||geo.query.yahoo.com^
||onepush.query.yahoo.com^
||bats.video.yahoo.com^
||visit.webhosting.yahoo.com^
||ads.yap.yahoo.com^
||m.yap.yahoo.com^
||partnerads.ysm.yahoo.com^
# Yandex
||appmetrica.yandex.com^
||redirect.appmetrica.yandex.com^
||19534.redirect.appmetrica.yandex.com^
||3.redirect.appmetrica.yandex.com^
||30488.redirect.appmetrica.yandex.com^
||4.redirect.appmetrica.yandex.com^
||report.appmetrica.yandex.net^
||extmaps-api.yandex.net^
||analytics.mobile.yandex.net^
||banners.mobile.yandex.net^
||banners-slb.mobile.yandex.net^
||startup.mobile.yandex.net^
||offerwall.yandex.net^
||adfox.yandex.ru^
||matchid.adfox.yandex.ru^
||adsdk.yandex.ru^
||an.yandex.ru^
||redirect.appmetrica.yandex.ru^
||awaps.yandex.ru^
||awsync.yandex.ru^
||bs.yandex.ru^
||bs-meta.yandex.ru^
||clck.yandex.ru^
||informer.yandex.ru^
||kiks.yandex.ru^
||grade.market.yandex.ru^
||mc.yandex.ru^
||metrika.yandex.ru^
||click.sender.yandex.ru^
||share.yandex.ru^
||yandexadexchange.net^
||mobile.yandexadexchange.net^
#Google Analytics
||google-analytics.com^
||ssl.google-analytics.com^
#Hotjar
||api-hotjar.com^
||hotjar-analytics.com^
||hotjar.com^
||static.hotjar.com^
#MouseFlow
||mouseflow.com^
||a.mouseflow.com^
#FreshMarketer
||freshmarketer.com^
#Luckyorange
||luckyorange.com^
||luckyorange.net^
||cdn.luckyorange.com^
||w1.luckyorange.com^
||upload.luckyorange.net^
||cs.luckyorange.net^
||settings.luckyorange.net^
#Stats WP Plugin
||stats.wp.com^
##Error Trackers ====================
#Bugsnag
||notify.bugsnag.com^
||sessions.bugsnag.com^
||api.bugsnag.com^
||app.bugsnag.com^
#Sentry
||browser.sentry-cdn.com^
||app.getsentry.com^
##Social ====================
#Facebook
||pixel.facebook.com^
||analytics.facebook.com^
||ads.facebook.com^
||an.facebook.com^
#Twitter
||ads-api.twitter.com^
||advertising.twitter.com^
||ads-twitter.com^
||static.ads-twitter.com^
#LinkedIn
||ads.linkedin.com^
||analytics.pointdrive.linkedin.com^
#Pinterest
||ads.pinterest.com^
||log.pinterest.com^
||ads-dev.pinterest.com^
||analytics.pinterest.com^
||trk.pinterest.com^
||trk2.pinterest.com^
||widgets.pinterest.com^
#Reddit
||ads.reddit.com^
||rereddit.com^
||events.redditmedia.com^
||d.reddit.com^
# TikTok
||ads-sg.tiktok.com^
||analytics-sg.tiktok.com^
||ads.tiktok.com^
||analytics.tiktok.com^
#YouTube
||ads.youtube.com^
||youtube.cleverads.vn^
##Mix ====================
# Yahoo
||ads.yahoo.com^
||adserver.yahoo.com^
||global.adserver.yahoo.com^
||us.adserver.yahoo.com^
||adspecs.yahoo.com^
||advertising.yahoo.com^
||analytics.yahoo.com^
||analytics.query.yahoo.com^
||ads.yap.yahoo.com^
||m.yap.yahoo.com^
||partnerads.ysm.yahoo.com^
# Yandex
||appmetrica.yandex.com^
||redirect.appmetrica.yandex.com^
||19534.redirect.appmetrica.yandex.com^
||3.redirect.appmetrica.yandex.com^
||30488.redirect.appmetrica.yandex.com^
||4.redirect.appmetrica.yandex.com^
||report.appmetrica.yandex.net^
||extmaps-api.yandex.net^
||analytics.mobile.yandex.net^
||banners.mobile.yandex.net^
||banners-slb.mobile.yandex.net^
||startup.mobile.yandex.net^
||offerwall.yandex.net^
||adfox.yandex.ru^
||matchid.adfox.yandex.ru^
||adsdk.yandex.ru^
||an.yandex.ru^
||redirect.appmetrica.yandex.ru^
||awaps.yandex.ru^
||awsync.yandex.ru^
||bs.yandex.ru^
||bs-meta.yandex.ru^
||clck.yandex.ru^
||informer.yandex.ru^
||kiks.yandex.ru^
||grade.market.yandex.ru^
||mc.yandex.ru^
||metrika.yandex.ru^
||click.sender.yandex.ru^
||share.yandex.ru^
||yandexadexchange.net^
||mobile.yandexadexchange.net^
##OEM ====================
#Realme
||bdapi-in-ads.realmemobile.com^
#OPPO
||adsfs.oppomobile.com^
||adx.ads.oppomobile.com^
||bdapi.ads.oppomobile.com^
||ck.ads.oppomobile.com^
||data.ads.oppomobile.com^
||g1.ads.oppomobile.com^
#Xiaomi
||api.ad.xiaomi.com^
||app.chat.xiaomi.net^
||data.mistat.xiaomi.com^
||data.mistat.intl.xiaomi.com^
||data.mistat.india.xiaomi.com^
||data.mistat.ru^s.xiaomi.com^
||sdkconfig.ad.xiaomi.com^
||sdkconfig.ad.intl.xiaomi.com^
||globalapi.ad.xiaomi.com^
||www.cdn.ad.xiaomi.com^
||tracking.miui.com^
||sa.api.intl.miui.com^
||tracking.miui.com^
||tracking.intl.miui.com^
||tracking.india.miui.com^
||tracking.ru^s.miui.com^
#OnePlus
||analytics.oneplus.cn^
||click.oneplus.cn^
||click.oneplus.com^
||open.oneplus.net^
#Huawei
||metrics.data.hicloud.com^
||metrics1.data.hicloud.com^
||metrics2.data.hicloud.com^
||metrics3.data.hicloud.com^
||metrics4.data.hicloud.com^
||metrics5.data.hicloud.com^
||logservice.hicloud.com^
||logservice1.hicloud.com^
||metrics-dra.dt.hicloud.com^
||logbak.hicloud.com^
#Samsung
||ad.samsungadhub.com^
||samsungadhub.com^
||samsungads.com^
||smetrics.samsung.com^
||nmetrics.samsung.com^
||samsung-com.112.2o7.net^
||business.samsungusa.com^
||analytics.samsungknox.com^
||bigdata.ssp.samsung.com^
||analytics-api.samsunghealthcn.com^
||config.samsungads.com^
#Apple
||metrics.apple.com^
||securemetrics.apple.com^
||supportmetrics.apple.com^
||metrics.icloud.com^
||metrics.mzstatic.com^
||dzc-metrics.mzstatic.com^
||books-analytics-events.news.apple-dns.net^
||books-analytics-events.apple.com^
||stocks-analytics-events.apple.com^
||stocks-analytics-events.news.apple-dns.net^

 

[Coop777]

98 %, Need to get my pi-hole container upgraded soon.

 

[Telos]

The list from the website ready to be copied to AdGuard

That's cheating and just "beating the system".

Interestingly though, when I added the few sites that were reported unblocked, and re-ran the test... a few more sites appeared. So I reverted to the rule set I use (as initially tested).

 

[WST16]

That's cheating and just "beating the system".

Interestingly though, when I added the few sites that were reported unblocked, and re-ran the test... a few more sites appeared. So I reverted to the rule set I use (as initially tested).

I don’t think it’s cheating and beating the system. That’s the essence of the system at its core— blocking rules.
Without adding all the rules, I think it's a hit and miss depending on the browser you're using and the DNS configured.

I've added all the rules to AdGuard's custom filters and I have AdGuard's Family DNS configured and 100% gets blocked on the LAN. I did the same on my AdGuard pro on my iPhone and it's 100% too.

Instead of adding a few, add them all and test. Removing them is a simple highlight all and taping delete if you decide that you don’t want them for any reason.

 

[Telos]

Here’s an installation video (referenced initially by @Telos) and I found it to be the best.

Just to update... I did use this video guide in my initial system, but later reverted and chose to run without MacVLAN. I never noticed the difference. Since then, I've off loaded AdguardHome to an Rpi.

 

[WST16]

Glad it’s working for you.

Initially, I tried PIhole on my NAS without macvlan and I think the caveat is that all logged ip address (clients) become the NAS IP address itself so I couldn’t setup polices per client. I assume AdGuard will behave the same. There was a way to overcome this without running macvlan if I recall and it was by running the pihole as a DHCP server I think, but I couldn’t do that because it needed something else that I couldn’t enable.

I moved to AdGuard with macvlan and I’ve never looked back, it’s been very solid.

 

[jeyare]

Pi hole (PH) running in macvlan, as primary DNS server for LAN environments (def in Unifi for each network when needed), local domain recognised in PH, DHCP by Unifi, dnscryp-proxy running directly in USG to Cloudflare, Cloudflare proxied, … and I can see every single LAN client records (query) in the PH.

No need to be worried about just 92% from the test. Because sometimes we need some services related to the blacklist records, e.g. CDN route to gstatic.com from few useful Google services.

Then I found yesterday that the priority order defined by PH architecture:
1. Exact whitelist
2. Regex whitelist
3. Exact blacklist
4. Blocklist domains (aka Gravity and custom add lists)
5. Regex blacklist
doesn’t work 100% reliable.
Single domain from the exact whitelist was blocked by another in exact blacklist. Due the “hidden” CDN stream.

but still it is really simple, easy and useful solution, means the PH

Now working at SQLLite ODBC connector into PowerBI to get more analytics than from Influx/Telegraf/Grafana. No need web UI for the visualisations. And also it is more secure.

 

[WST16]

No need to be worried about just 92% from the test.

I block them all until something breaks and then I open what’s needed if it’s a must

 

[Coop777]

a little off topic here..but is anyone using pi-hole for DHCP? I've opened up relevant ports for it but for some reason it doesn't seem to "work"

I as well would like to see traffic by individual client IP address, but that doesn't work with router enabled DHCP. The tutorial I followed for setting it up didn't reference specific setup for DHCP so there may be something I am missing.

 

[WST16]

a little off topic here..but is anyone using pi-hole for DHCP? I've opened up relevant ports for it but for some reason it doesn't seem to "work"

I as well would like to see traffic by individual client IP address, but that doesn't work with router enabled DHCP. The tutorial I followed for setting it up didn't reference specific setup for DHCP so there may be something I am missing.

Are you running it with its own ip address using macvlan? If not, you might want to try that and see if it works.

The same guy did a tutorial for pi hole using macvlan. If I were you I’d shut down the current pi hole and try this one. If it works you can remove the old one, if it doesn’t, remove this and bring up the new one.

 

[jeyare]

to see client’s records in PH:
1. NAS: mcvlan + separate bridge network
2. the mcvlan IP you will use as the PH DNS IP for entire environment
3. in the PH DNS setup: uncheck all the Upstream servers, set new Upstream server = your router IP. Enable Listen on all interfaces. When you don’t have DNSSEC running in router, then you can enable the service in PH. And finally Conditional forwarding: Local network .. set your LAN subnet, IP of your DHCP server (router), local domain (up to you). Repeat for each LAN subnets.
4. Your router setup (when this is your DHCP server). Setup for WAN interface: in my case it’s Cloudflare range of DNS IPs. For LAN interfaces (each LAN netw/VLAN) you need define just the PH DNS IP.
Done
You will see all the clients records, clearly.

I have dnscrypt proxy running in Unifi USG, what is more flexible from each possible setup option.
OFC you need enable DNSSEC in your registar, when used diff cert than LE, e.g. wildcard in my case. Plus you need setup in the opposite DNS server, e.g. Cloudflare.

 

[jeyare]

when you like to use more than static control of your network = Pi-hole or AG. You can plug the data from PH SQLLite in realtime analytic hub.
Then you need WinOS, SQLLite 3 ODBC driver and something like duo PowerBI desktop + Python 3.x./Jupyter DataScience pack (in your container).
When you combine this base with tcpdump results, you can definitely track your LAN traffic up to specific source level (application, service). Better than host only.
So the first level or the base is done.
You can flexible investigate specific time of events or similar behavior of LAN host/s.
What is great, that up to 95% of the blocked traffic is captured by Gravity = well maintained source. OFC, it depends on defined policies and restrictions at client’s side.