Overblocking in a Web 2.0 World

In today's Web 2.0 world, the concept of a web page is kind of a misnomer. Most are already aware that a single web page is actually made up considerably more embedded links, and in some cases hundreds of embedded links providing information to display a single unified page. Any one of those hundreds of links could contain malware, while the other hundreds of links could contain information necessary to complete an organization's users task or job at hand.

For most, the secure web gateway is the device in the network that handles protecting the end-user from the malware by blocking the specific embedded URL that contains the malware. But often it's not that simple. In today's sophisticated attacks, which take advantage of SEO (Search Engine Optimization) poisoning and link farms, where tens of thousands of links are created to a few handfuls of malware sites, it's hard for security to devices to determine where the good sites are and which sites may just contain an embedded link to a malware site (often hosting good content at the same time). The challenge is of course not to block the websites that only contain links to other links that contain links to malware. Blocking at a level that's too high will inadvertently cause end-users to miss content they need, and produce an effect known as over-blocking.

One of the problems with over-blocking is that it may make your secure web gateway solution look like it's doing a great job, but without doing the work to see if there really is a malware on a link, you don't know if your solution has just prevented you from reaching important information. While over-blocking is a well known problem it's harder to determine whether it's occurring until an end-user complains about access to information. Part of the test in finding out whether your secure web gateway solution is over-blocking is finding out what they do to prevent over-blocking. Understanding how the solution works, and what causes a site to be blocked is the first step in preventing over-blocking in a Web 2.0 World.