Out with the Old, In with the New

A little technology based spring cleaning - my thoughts on malware and e-mail security technologies that are being phased out and the newer, better technologies that are taking their place:

The first technology being replaced is the traditional spam filter. As we all know, the % of spam and unwanted e-mail has been steadily increasing for the last decade. Clearly, the concept of filtering e-mail based on content has failed to even make a dent in the problem. Even the best spam filters, all based on a blend of heuristics and/or Bayesian probability, have proven to be reactive and ineffective. The replacement for these failed systems is e-mail address verification. Only through the establishment of human-to-human e-mail address verification can people begin to regain their confidence in e-mail as a trusted form of communication for business.

The next technology being replaced is signature based anti-virus tools. Gone are the days when enterprises can rely on a single vendor, or even a group of vendors, to provide virus definitions and scanning tools. Intelligent pattern recognition engines, like those provided by Commtouch RPD, are proving to be more effective and more efficient than traditional signature based tools.

Next, all technologies that call themselves "IP Reputations" technologies are on their way out. Products and services like Spamhaus, Spamcop, and other blacklisting databases are the cancer of the Internet and cause more harm than good. As the world moves from IPv4 to IPv6, these IP Reputation tools/systems/databases will become irrelevant.

Finally, from a security perspective, IPv4 is on its way out, to be replaced with IPv6. This transition is well underway within the confines of wireless networks and large private networks, and will soon be making its way into the public domain.

...and who says e-mail spam filtering works?

The following excerpt comes from MSNBC's "The Red Tape Chronicles" :

[Let me begin by saying that you cannot make this stuff up!]

Friday: 10 Oct 2008
(http://redtape.msnbc.com/2008/10/att-customer-ca.html#posts)

AT&T reserves the right to change its terms of service by sending its Internet service customers an e-mail. Apparently, it also reserves the right to deposit those e-mails into its customers’ junk mail folders.

Last month, AT&T made some controversial changes to its Internet policies. Verbiage indicating that high-bandwidth users might experience some intentional slowdowns irritated some techies; another section that forces customers to use binding arbitration to resolve disputes annoyed consumer organizations; and an L.A. Times reporter bristled at the size of the full new agreement -- 2,500 pages.

But Lance Mead, an AT&T Internet customer from Encino, Calif., almost missed the entire controversy. His notification of the new terms of service was sent via e-mail on Sept. 18, but AT&T's own spam filters trapped the e-mail as spam and deposited it in his junk mail folder, he said. On a whim, he checked the folder and spotted the notice. He was furious.

Someone -- anyone -- please tell me how this is not proof positive the entire premise behind e-mail spam filtering is seriously flawed? I completely understand that mistakes happen. However, these "mistakes" are also considered "false positives." In the "e-mail game" it is the false-positives that cost business real money. Is it really the end of the world if 5% to 10% of the e-mail received in your inbox is spam? Probably not. It is unnecessary, annoying, and unproductive to be forced to wade through spam, but missing an important e-mail thanks to the flawed concept of filters, a.k.a. guessing machines, should be considered absolutely unacceptable.