Obfuscating the URL

Common techniques used by spammers seems to also cause grief for many proxies that scan URLs against URL database lists. Spammers would try to hide the actual URL in an email by using some standard features of URLs. In addition to domain names, URLs can contain usernames and passwords, IP addresses, and encoded IP addresses. All of these provide ways to create URL's that look like they are going to a site, but really are going somewhere else.

Simple proxies don't recognize anything other than a standard domain name. So many of the techniques in the linked article above will get past a URL filter. So if someone wanted to bypass a typical proxy they could obfuscate the URL they type in the browser and still get to sites that are deemed inappropriate based on policy or contain malicious content.

Many of the better proxies are aware of these techniques and will "translate" the URL before passing it through the URL filter to ensure the real URL gets filtered, and blocked or scanned if appropriate.

Be sure to include obfuscation techniques in your evaluation of any proxy solution for your organization.