As a Beta tester for MailWasher, among other applications, I have been testing these filters for over a week now. While I appreciate all the hard work that went into them, I am terminating my test and uninstalling the entire Wizcrafts filter set. My conclusion is the Wizcrafts Filters add no value, falsely tag an unacceptable level of legitimate emails as spam, and actually increase workload and decrease user productivity.

Note the following:

• I had to delete the following filters out of the set because they repeatedly tagged legitimate email from various places in the US as spam:
  
  • Blocked Countries
    
  • APNIC (Asian-Pacific)
    
  • Pill Filter
    
  • HTML Tricks (tagged multiple newletters, including the latest from Zonelabs)
    
  • Misspelled Drug (this actually tagged today's Mailwasher Pro summary for the last week! )
    
    Fortunately, I had already gone through the filter set and changed all the automatic bouncing, blacklisting and deleting or I would have lost important legitimate emails! See below for more details.
  
  
• I have just two of my own filters. One tags my SpamCop reporting emails, and the other tags spam notifications from my ISP. The Wizcrafts filters have yet to be the sole method used to identify real spam - that is, all spam identified by Wizcrafts filters were also identified by MailWasher's own Learning Filter, DNSBL servers, and/or my 2 previously mentioned simple filters. In other words, they would have been caught anyway, without placing any performance burden on MWP.
  
  
• Since using the Wizcrafts filter set, I continue to get untagged spam at about the same rate, 1 or 2 out of 100/day - the same with or without the Wizcrafts filters. They added no advantage/value.
  
  
• I reported to my colleagues earlier in the week my displeasure at seeing the bounce feature frequently, though inconsistently, used throughout the feature set. I am happy to see that the bounce feature was removed from the filter set on Aug 8 - although the reason given in the filter set is not entirely accurate. Yes, the address is usually forged, meaning the bounce will go to the wrong (likely innocent) address, but bouncing also adds more unwanted email on the already crowded Internet backbone, wasting even more bandwidth. Bouncing email is also a good way to get your address added to blacklists! For anyone creating their own filters, see What is Wrong With Bouncing Spam to see why bouncing spam is a bad idea.
  
  
• There are many other inconsistent use of options that make managing the filter set difficult at best - especially considering that NO spam filter or spam blocker program is perfect, and there can be no guarantee of no false positives. For example, within the 9 Image Spam filters, some set the spam for Hidden, some not. Some set the spam for Automatic, some do not. The inconsistencies make understanding what is happening difficult.
  
  
• The use of "Automatic Process" is unacceptable! You should NEVER use filters set to automatically delete a message until you are 100% certain no false positives/legitimate email will be irretrievably lost.
  
  
• Automatically hiding email is almost as dangerous as automatically deleting. Unless you are sure what the filter is hiding, don't hide.
  
  
• There is an apparent haphazard and inconsistent use of the "Blacklist" feature. It is important to understand that many experts agree blacklists have limited effectiveness in detecting spam. This is because even wannabe badguys dump addresses after a day or two as they are aware that once the address is known, it will be worthless. So adding a worthless address to your blacklist simply clutters up your blacklist and Mailwasher. For this reason, I do not use, nor do I recommend the use of blacklists. I do use white/friend lists, as changes to that are infrequent and easy to manage.

_________________
Bill, AFE7Ret
Freedom is NOT Free!

Bill;
Thanks for your astute input. I realize that some of my filters need to be readjusted and I have been working on that, one at a time, in between other pursuits. I will remove automatic deletions, blacklisting and hiding from the public filters I publish and get them online, hopefully this weekend. The reason there are so many auto delete filters is because that's how I use them and they are extremely effective for me. When I update a filter rule I normally copy and paste it into the published list as is. I will undertake to cleanse them of automatic deletions and blacklisting in the future. As you noted I have already removed all bounce directives.

Concerning false positives, I get them occasionally as well and when I do I try to fix the bad rule that triggered the flag. The HTML Tricks filter is one that I work on frequently, which is why I moved it to the bottom of the list, in filters2.txt. It is still useful for detecting spam that the rest of my filters, and the learning filter have missed. That filter is not set to hide flagged messages, just to mark them for manual deletion.

You are correct about the increase in processing load and I have mentioned this several times. It is something I work on to the best of my abilities. I will continue to make my (sanitized) filters available for those who want to use them, as long as I use MailWasher Pro, which will probably be for a long time.

Thanks again for letting me know about these problems.

After considering Bill_Bright's comments and feedback from other MailWasher Pro users, I have just removed all instances I could find of the action directives "Automatic," "Blacklist," and "Hidden," in the MailWasher Pro Filters on my website. This includes both the long and short filter sets. I will keep one extra set online for those who prefer to have those (drastic) actions by default. Send me a PM for the URL to that set of filters.

I hope that this will minimize problems caused by false positives, as you will now have a chance to review the files that have been flagged as spam and decide if they really are spam. If not, mark them as legitimate and add them to your Friends List.

If you find that some of my filters are indeed 100% correct in ID-ing spam, edit the actions as you wish, from within the MailWasher Pro interface, via the Filter Sidebar. Use Control + F7 to display the sidebar, then click on the Filters tab.

Thank you for addressing these issues so quickly!

It is important for all to understand and accept that total elimination of all false positives is an impossible goal. And it is also impossible for any spam blocker to correctly tag every piece of spam, as spam, 100% of the time. Therefore, it is inevitable that we will occasionally get a false positive, and we will also occasionally get spam that makes it past all the filters. That's the beauty of MailWasher; unlike all other spam blocker programs, the unidentified spam that makes it past all the filters still remains on the server, and is not downloaded on to your PC and already in the front door, if you will.

Many of us consider email an invaluable communications tool and the risk of losing even a single email from an important client or contact is too great. It is because of those two unobtainable goals that automatic processing/deleting of email should never be done. Of course, if you don't consider your email important and don't care if you lose some of it, those may be acceptable risks.

But then there are always exceptions. If you want to automatically delete all email from your X's email address, for example, you can create a filter that addresses that specific parameter. But, when dealing with variables, such as these filters, that's where automatic processing/deleting becomes an issue.

I think with the changes you have made, you eliminated the dangers of the filters. Added value vs added workload managing them is still a factor that one would have to weigh. Although your filters certainly identified most of the spam I received, as I mentioned before, the other spam tools within MWP tagged the same ones too.

That said, once again, I appreciate all the hard work that went into compiling the set and I will try them again soon. If nothing else, seeing the Junk Mail pie chart breakdown in statistics is interesting.

_________________
Bill, AFE7Ret
Freedom is NOT Free!

I've been using these filters now for about a day. By removing the HIDDEN and AUTOMATIC directives, I was able to examine the effects of your filters.

I have been very impressed at its ability to remove ALL of the newer spam types. In my limited time, I have received NO false positives, and NO spam messages have made it through the added layer provided by your filter.

I am trying to decide whether to delete any of my own filters due to the superior coverage of yours; however, with 15 email accounts and an average of 300-400 spam mails received daily, the extra processing time added by the new filters with their regular expressions are adding about 3 minutes to the already lengthy processing time each time I read email.

That may also be affected by my decision to scan the maximum number of lines per email, and keep my deleted email records for the maximum allowable time.

Edit: My statement about false positives was a bit incorrect. I happen to correspond with a few people in Asia and the Middle East, but hadn't actually added them to my friends list. I needed to do that after installing your filter due to the country filtering.

Thanks for the update. I saw the ebay filter and agree with your assessment. As for viruses, where specific signatures can be determined, I don't have a problem with them either - not since you removed the automatic processing settings.

Thanks for your understanding on this. I must say your receptiveness to my rather to-the-point criticisms is much appreciated. I think once you complete your review and make the necessary changes to make the feature set "defaults" consistent throughout, users will then be confident they understand exactly what is happening their email when they first start using your filters. And from there, they can tweak and customize as they see fit.

_________________
Bill, AFE7Ret
Freedom is NOT Free!

Thanks Bill, I appreciate your encouragement. I guess I spend too much time under the hood tinkering with the timing to appreciate the problems others might have driving this hotrod. I'll keep at it!

I just finished refining the (virus/worm) various attachments filters and trimmed them down to only 4 rules. I have tested them and they seem to work as designed, but I'm open to suggestions after you test them. Note that I have kept the Friends List override on all 4 if only to alert the user that somebody has sent him/her an attachment that is a .exe, or .zip, or .rar. or .gz. There is no action preselected for the two filters dealing with those file types. OTOH. there are two filters that detect truely dangerous attachments and combinations of extensions, and these are marked for (manual) deletion.

The reason I left the Takes Precedence option checked is because viruses and Worms still arrive with a From address or domain that many people may have on their friends lists, or even from a forged account on your own domains. I am open to arguments against having these overrides in place, for some or all of these 4 filters, but it seems prudent to me.

I'm finished for tonight and will take up the project tomorrow. zzzzzzzzzzz

Not necessarily anything new, but I thought I'd add my perspective to the discussion on filters, etc.

First, I rely heavily on the Bayesian filters in the Learning Tool. I figure 95% of my selections are taken care of before I even see the emails. My only filters are some specifics to keep multiple addresses that really belong to one organization together - so I can see who it is in the Status column, by name, and the filter to allow retrieval of emails MW has deleted, plus firends and (a very shortlived) blacklist. I then do a quick scan of the sorted emails with most of my attention paid to the ones not marked for deletion. I also use PopFile, but I remember a mention earlier on, I think in this thread, about dumping all the spam/viral/any other junk at the ISP's server. That adds greatly to my peace of mind. If a piece of junk of the viral kind makes it to my computer, I'm not happy. That's too close. Hopefully PopFile will then ID it, but I'm not about to open it to find out if it is really viral. I won't even do that if the email is on the ISP server, since opening the email can, in some cases, initiate the virus. At least, if it's id'ed on the server I won't ever even have it on my hard drive. I really only use PopFile as a truly last resort to separate emails I'd like to decide on versus friends and those I definitely want to read. By the time they get to PopFile, the numbers are much smaller (approx 10% of what shows up at the server), and I don't mind manual selection.

I have to add: I'm a retired programmer/analyst and have done my share of code. But I don't even like Regex. I didn't when I had to use it professionally. Because I'm retired, I have the time to manually refine my email filtering. And since I'm only using MWP on one account, I'm not taxing anything very much. I know others have lots more email and need to automate as much as possible, but I agree with everything said about FP's and the need for some refined selection rather than automatically deleting anything. If I used email to do a part of my business life, I would never have anything auto-deleted. Too much of a chance a piece of vital email could go missing without me knowing it. If there are dollars associated with that email, woe is me. I'm the only one I want making a decision like that.