Fighting Junk Mail with Netscape 7.1

Summary: Is your mail account drowning in a flood of spam? Netscape 7.1 includes built-in junk mail filtering that you can train to meet your specific needs. Eric Meyer share his experiences with making the junk mail controls more effective, and looks at other preferences you might want to set to keep your account safe.

One of the great things about Netscape 7.1 is that it includes the powerful junk-mail (a.k.a. "spam") filtering introduced in Mozilla 1.3. This filter technology is based on Bayesian analysis of incoming content, and as such can be trained by the user to meet individual spam-fighting needs. This ability can replace the use of server-side filtering, or can be used in concert with such filtering.

During the testing of internal NS7.1 builds, I found that there were certain filter settings that sped up the process of training the filters while avoiding the deletion of legitimate e-mail. If you're upgrading to the Netscape mail client, or thinking about switching, you may find this look at what I discovered useful. I've definitely become spoiled by the technology: within just a few weeks, the amount of spam slipping past the filters was down to almost zero, and I was getting all of my legitimate mail. Combined with the popup blocking built into Netscape since 7.01, Netscape 7.1 makes both the Web and e-mail fun again.

Laying the Foundations

When you first launch Netscape 7.1, its junk-mail filtering controls are already turned on. You can access the controls through the "Junk Mail Controls..." entry in the "Tools" menu. Here's what the default settings will do for your accounts:

• Run junk mail filters
• Avoid marking as junk mail from people in your address book
• Automatically move mail the filters mark as junk into the "Junk" folder
• Automatically move mail the user manually marks as junk into the "Junk" folder

Right off the bat, that looks pretty darned good, but there's a way to improve the settings. I advise that you change that third setting however, and disable the moving of mail that the filters mark as junk.

Why? Because, when you first start filtering, the rules aren't tuned yet. They're not bad at separating the junk from legitimate mail, but they aren't great either. My rough estimate of the initial filter accuracy was in the 80% range, with different accounts having different success rates. That's definitely better than nothing, but it also means that about 20% of your mail will be misclassified: junk will end up in your inbox, and legitimate mail will end up in the Junk folder (these are known as "false positives").

Fine-Tuning the Filters

False positives aren't a major tragedy, since you can always go into the Junk folder and rescue legitimate mail. All you have to do is mark it as being Not Junk, and move it into the folder where it's supposed to have gone. This can get tedious, though. I found it a lot more efficient to let the mail client spend a few days marking mail as junk, but not actually moving it anywhere. This lets you review the messages in place, adjust the junk classification as needed, and then drag all the junk into the Junk folder. A handy way to speed this process is to use the "View" dropdown at the top of the message list. You can select "Junk" so that you see only the messages in the folder that are marked as junk. This lets you make last-minute changes and then just drag the whole collection into the Junk folder.

During this phase, it's up to you whether you want to leave the "move mail I mark as junk into the Junk folder" option enabled. It can be disconcerting to have mail disappear the moment you mark it as being junk, and if you ever mark the wrong message as being junk, you have to go digging for it in the Junk folder. On the other hand, it does get the actual junk out of your way faster.

It was my experience that after about a week of watching the junk filters closely, they were approaching 99% success rates, with almost no false positives (legitimate mail misidentified as junk). Another week or so of this yielded no major changes in the success rate, although the false positives droped to basically zero.

Automatic Junking

It was at this point I decided to re-enable the option to automatically drop mail marked as junk into the Junk folder. The hope was that I would be faced with no more spam in my Inboxes, and could just go into the Junk folder periodically to check for any lingering false positives. With some trepidation, I went ahead and did it.

Over the next few days, I found one or two pieces of junk got past the filters each day. Given that my pre-7.1 spam levels were in the area of 50-100 messages per day, this was a huge relief. I went into the Junk folder two or three times a day, checked to make sure there had been no false positives, and marked the new spam as being read. This let me quickly glance at my folder listing and see how many new junk messages had come in.

What I found was that my false positive rate was effectively zero. There was one message that came in during that period that was misidentified as junk. It came from someone with whom I had never before corresponded, and the subject line, to be honest, was rather spammish ("Advice needed," which is very close to the subject line of several Nigerian mining scam variants). With that one exception, every piece of mail identified as junk was in fact junk.

A few more weeks of running with this combination yielded the same basic results: a couple of junk messages evading the filter each day, and no false positives.

As I write this, I'm considering taking another step forward, which is to automatically delete mail that's been in the Junk folder longer than a certain period of time. At this point, I feel the spam filters are good enough that I can set it to 7 days, which gives me enough time to review for false positives (I'm paranoid about losing legitimate data) ever few days or so, while saving on disk space. Deleted junk mail doesn't harm the filters, so the only possible downside is accidentally letting real mail get past me for more than a week. Over time, I will probably lower the deletion period to five days, then three, then one.

Another way to accomplish the same effect would be to move mail marked as junk into the Trash, instead of the Junk folder. I find it more useful to keep my junk in its own folder, since that means anything int he Trash is mail I manually deleted, but your preferences may vary.

Related Settings

While we're on the subject, there is another set of preference settings you might want to consider: the "Return Receipts" section of the "Mail & Newsgroups" preferences for your account. You can elect never to send return receipts, of course, which will keep your account from ever notifying a spammer that your account exists. If that's too ham-fisted for you, then you can assert a little more control. I'd definitely recommend setting all three to "Ask Me" at the most, and possibly mixing up "Ask Me" and "Never Send Receipt." As an example, I have my preferences set to never send a return receipt if my name doesn't appear in the To: or Cc: lines of the message. For all other messages, I have the client ask me if I want to send a return receipt or not.

Just as important to hiding from spammers is the disabling of cookies, images, and scripts in the mail client. This has to be carried out using three different preferences dialogs, but it's worth the effort. In the general "Preferences..." dialog, visit the following three panels:

• Privacy & Security
  • Cookies
  • Images
• Advanced
  • Scripts & Plugins

In each of these panels, there are options to enable or disable these features in mail and newsgroups. Make sure all three are disabled for maximum security. As an example, many spammers use image URLs as a way of detecting an active e-mail account. If you load one of these images, the spammer's server will record that the image was loaded and decipher the account that received that particular message. The only way to avoid this is to avoid loading images in the first place. In a similar fashion, JavaScript execution in mail messages is far too dangerous to be trusted, and you should make sure it is disabled no matter how much or little spam you receive.

There is one last thing to note, which is that while you can delete junk mail without harming the filter training, you can't delete the filter file (<tt>training.dat</tt>, stored in each profile folder) without resetting your filters to their initial state. If you do this, you'll have to go through the whole training process again. Of course, should that happen, you probably won't lack for training material.

Conclusion

Although it took a couple of weeks to get the Netscape mail client's junk mail filters really fine-tuned, it was two weeks wisely invested. As I write this, my spam problem has dropped to almost nothing, and I'm far more confident that I'm not losing legitimate mail amongst the avalanche of spam that usually filled up my inbox. Because the options are flexible, I feel like I really have control over my mail, possibly for the first time in years.