If anyone is interested, I have a new RegExp filter that can be used to spot those messages where the Spammers try to obfuscate their SPAM using ASCII charactor codes. (&# 033 etc)

This Expression will look for any ASCII charactor that would normally not need to be encoded...charactors 33-126 listed as follows:

I've been running a filter like this for several weeks now but have to date only had a single hit (# of encoded characters set to 15).

Ikeb,

Is it limited to the charactors that would normally not require encoding?

If someone is encoding charactors that don't require encoding (which is what the filter I posted looks for) it is a BIG FLAG that it is SPAM.

A lower threshold than 15 can be used in that case. If you are looking for any encoded charactor....you probably do need a higher threshold to prevent false positives...and will have a less effective filter.

Ikeb,

For the last few weeks I have left extended error logging on so that I can see what was in the messages that my filters were auto-deleting.

One of the things that I have noticed is an increase in the kind of obfuscations that the filter that I am targeting is going after.

Right now it might only trap about 5% of the incoming SPAM...but as that kind of trick seems to be on the increase, it might be a good idea to get the filter ready for it.

I am not posting all of my filter strategies because I don't want to give too much to the Spammers who may be monitoring this forum.

My IANA-Reserved filter was one that I posted here, partly as a help to all....but also as a 'plant' to see if this forum was being watched.

That IANA-Reserved filter has been steadily decreasing in hits since I posted it here. Before I posted it, it was trapping more than 35% of all incoming SPAM....now it is trapping less than 10%. That filter has remained at the top of my list so it is the first thing that will trap anything that it is aimed for.

I have been studying the messages that have been auto-deleted and although the number of messages trapped by IANA-Reserved has been decreasing, the number of messages that have forged Received: lines in the header has not been.

This tells me that it is not the NEW Anti-Spam Law...and it's provisions for criminalizing the forging of header information that is responsible for the decrease in hits by IANA-Reserved. It is more than likely the compromise of the strategy to the Spammers.

I am willing to accept the loss of IANA-Reserved as an effective filter if the Spammers have responded to it. I will not accept the loss of certain other filter strategies though.

In the case of the ASCII encoding filter...I look at it this way. I want the Spammers to know that there is a means to trap that kind of obfuscation.

That is why I posted it.

It may not trap a lot in the future....but it may prevent a lot of that from being sent in the future also.

Denn hits on the key problem with static filters, if you make them public the spammers will use them to test their outgoing e-mail until it passes before doing a spam run. Not sharing will make your filter more effective for longer unless it is generic enough that it can't be bypassed and that is almost impossible to do.

First Alert faces the same problem, spammers are probably on the beta testing team and on the current pre-release version working like the dickens to find a way to get past the system.

Baysean filters are usually unshared or only an initial set of data is shared and the user then further trains the filter system. The result is that a spammer can't train his system to beat your filters since he has no access to them.

Stan,

One of the things that I did not like about the whole idea of CFS was that the Spammers will probably be monitoring how the system works by subscribing to it themselves.

The one thing that the system may have going for it is that only a certain amount of the 'fingerprint' strategy would need to be incorporated into the software that the user installs on their machine.

If Firetrust does it right, they could have one hell of an anti-spam system going with 'First Alert'. If they place too much of their strategy into the user's software....they are going to pay a price for it.

As far as sharing filters goes...I may not post all of my filter strategies here....but I will help out anyone I can....as much as I can....to develop any strategy they may suggest here.

I have seen some pretty nice strategies suggested by other users on this forum...some of which could be compromised by posting here, and some that would be unaffected.

The trick for keeping many of these filter strategies safe usually lies in the variations that can be developed. Once you have a filter strategy developed, you can post some of the basics....but keep the many possible variations of those basics to yourself. In that way you can help out as many other users as possible...and still keep the Spammers clueless as to what you are actually doing.

The people at the bottom of the filter food chain will derive some benefit from these posts, though not as much as those who will take the time to really look at and understand the filters, and their underlying stategies.

For those that do take the time, they will get the maximum from the filters posted to this forum.


But...They will have to figure out many of those higher level strategies on their own.


By the way...Ikeb is a fast learner. If he plays his hand right, he will be able to counter just about anything the Spammers might come up with. I think he realizes the difference between what tools and strategies he can safely put on the table....and what he should keep to himself.

I'm just guessing but I'd be surprised if the folks writing the spamming software don't have a list of features that includes all the spam tools and filtering methods that it is designed to avoid.

Doing the research to avoid mailwasher would be too much for joe chickenboner but a major spam operation or a spam software vendor should find it well worth the effort.

Sure, I'll admit that's possible. But it still isn't going to make me "hold back" information that could help other users. I figure if everyone does the same, it will make life more difficult for SPAMers. By "holding back" it doesn't help me in the long run because I will have allowed SPAMers to successfully reach more email users, thus contributing towards even more SPAM in the future.

I didn't suggest that you hold back information on filters, only that you be aware of the risks of releasing the information.

For the filter savvy it shouldn't be much more work to take a filter that is working for them and that they want to protect and make minor changes to it that don't make much difference in its effectiveness and post that version.

For lots of filters like the URL blocking one I got help with (and that is still my biggest hitter) public release doesn't matter in the least as each user adds the data needed to make it work for them.

Oh how I wish it was $50 for a domain name, if you don't mind a spammy host you can bulk but .biz domain names for a tenth of that or less. Don't have the link handy but a google on bulk registry and biz might turn up some scary numbers.

I'd like to find a way to capture the spamcop spamvertized domain listing on a regular basis off their web page and plunk that into a filter. Even better would be a 7 day running list that you could update as often as you liked, sort of like the RBL listings work now.