Over the past three weeks, approximately 10 very popular websites have been miscategorized in our blacklists and content filtering categories. The most impact was seen when one of the data feeds powering DNSWatch blacklisted `download.microsoft.com`, `m.youtube.com`, and `www.youtube.com`. When these were added to our blacklists, the content at those sites was blocked and a number of alerts were generated. Once we analyzed these alerts and discovered the error, we immediately took action to rectify it. We apologize for any inconvenience this caused.
How Did This Happen?
DNSWatch regularly updates its internal list of bad domains from external sources. To avoid the situations above we stop common websites from being blocked, but our whitelisting approach does not always include subdomains. Likewise, if a subdomain is whitelisted, the parent domain is not always whitelisted. Thus access to `youtube.com` was allowed while `www.youtube.com` and `m.youtube.com` was blocked.
How Did You Fix This?
After noticing an increase in the number of alerts to legitimate domains and immediate feedback from our customers, our team took action by removing the offending domains from our blacklist. In tandem, we began re-assessing the quality of the original feeds that included these domains.
For the resource abuse category, we took the immediate action of using our content delivery network (CDN) data to whitelist popular CDNs and avoid them being miscategoried and erroneously blocked.
In the case of the Malware Corpus Tracker domain feed, we examined the domains in the feed looking for additional legitimate and benign services. Based on this analysis, we decided to temporarily disable this feed for all DNSWatch customers.
As of this afternoon, we have made changes to our whitelisting process to include subdomains of popular online services, which will prevent Microsoft, YouTube, and other popular services from being blocked in the future.
In the coming months, we will be reassessing our approach to automatically blocking malicious domains. The quality of a domain feed can change over time, and our current process doesn’t force us to reinvestigate feeds frequently. In addition, we will manually review daily changes to feeds, looking for both false positives and others signs in change of quality.
As always, we welcome your feedback. Please do not hesitate to email questions or suggestions directly to firstname.lastname@example.org
Again, we apologize for any inconvenience or interruption to your business this caused.
*** This is a Security Bloggers Network syndicated blog from Speaking of Malware | The Strongarm Blog – Strongarm Malware Protection authored by Stephen DiCato. Read the original post at: https://strongarm.io/blog/dnswatch-blocking-popular-websites/