A new filter set for MWP users brought to you by Wizcrafts!
Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
 CastleCops -> Mailwasher - Troubleshooting / General
Author: Wizcrafts, Location: Michigan PostPosted: Sun Nov 05, 2006 3:50 pm    Post subject:
Ikeb;
Am I allowed to post a link to the image filters on my own website, or should I copy and paste them into a forum reply?
Author: Wizcrafts, Location: Michigan PostPosted: Sun Nov 05, 2006 6:22 pm    Post subject:
Oh well, no reply, so here is a link to my custom MWP spam filter rules. The 2nd through 4th rules deal with the current crop of image spam for stocks and investments. Note that I have set them to automatically delete without notice. You may want to change that to "prompt," until you are certain that the rules are 100% accurate for your use.

Since the spammers occasionally alter their headers I alter my rules to match their techniques, usually within mere minutes of finding a new trick that gets past an existing filter. Such is the case with the GIF Spam #3 rule. However, even though they alter the layout there are always some things that must remain constant for their spam to work, and I always find those commonalities and create rules to detect it. I have a topic on my blog about this image spam issue and it gets a lot of hits.

I hope this helps other MailWasher Pro users. If the links are a problem I apologize in advance.
Author: stan_qaz,  PostPosted: Sun Nov 05, 2006 8:52 pm    Post subject:
Interesting collection of filters, Thanks for posting them.
Author: BuckeyeBasser, Location: Central Ohio, USA PostPosted: Sun Nov 05, 2006 8:54 pm    Post subject:
Be sure you close MailWasher completely before manually editing your filter file.
Author: Wizcrafts, Location: Michigan PostPosted: Sun Nov 05, 2006 10:00 pm    Post subject:
stan_qaz wrote:
Interesting collection of filters, Thanks for posting them.

You're welcome Stan, and other members! I'll post back here, or start a new thread, if the image spam coding vector changes and I create new rules to deal with it.

BTW: If you edit your own filters, there is a function test button for Regular Expressions. I use it against samples taken from the source code of actual spam messages to fine tune my filters. This is a great feature of the current flavor of MailWasher Pro.
Author: Wizcrafts, Location: Michigan PostPosted: Mon Nov 06, 2006 6:24 am    Post subject:
I just updated my MailWasher filters to intercept a new variation of the investments image spam. The new filter is GIF Spam #4. grab them at: www.wizcrafts.net/docs/filters.txt . Be sure to close MailWasher before editing filters.txt. Watch out for blank lines between or after rules, or extra spaces after the end of a rule. Either will cause you problems.
Author: Ikeb,  PostPosted: Mon Nov 06, 2006 6:28 pm    Post subject:
Wizcrafts wrote:
stan_qaz wrote:
Interesting collection of filters, Thanks for posting them.

You're welcome Stan, and other members! I'll post back here, or start a new thread, if the image spam coding vector changes and I create new rules to deal with it.

Sorry for the delay. I was busy elsewhere yesterday. Yeah I second Stan's sentiments, an interesting collection. Thank You I'd like to have it linked at the wiki if you don't mind -- perhaps featured as more current than Gary's filter collection? BTW, any connection to Gary's? Also, do any filters have to be tweaked to account for individual email addresses, etc.?

Wizcrafts wrote:
BTW: If you edit your own filters, there is a function test button for Regular Expressions. I use it against samples taken from the source code of actual spam messages to fine tune my filters. This is a great feature of the current flavor of MailWasher Pro.

Very limited though. It's much better to use Sorokin's TRegExpr when developing a filter.
Author: Wizcrafts, Location: Michigan PostPosted: Mon Nov 06, 2006 7:42 pm    Post subject:
Ikeb wrote:
Quote:

I'd like to have it linked at the wiki if you don't mind -- perhaps featured as more current than Gary's filter collection? BTW, any connection to Gary's? Also, do any filters have to be tweaked to account for individual email addresses, etc.?

Thanks for the good words!

I would be ecstatic to have my filters linked to in the Wiki! I have been refining my filters for several years and try to keep them mostly current. There are several rules that can probably be discarded, namely the Spam Domains #1 through #9 rules. Most of the domains listed are probably long gone by now.

I do believe that I have purged any specific personal email accounts from the list, and what remains does not require your email address to function. There are already plenty of examples about using custom filters that account for your own email addresses. I use several of those, but the rules I put on my website are much more effective for general spam, especially when BCC'd.

I don't have any connection with Gary. I appreciate his work on the filters a few years ago. I used them as the basis for my rules.

Thanks for the link to the RegExpr tester.

As I mentioned earlier, the spammers must be reading my posts because they are altering certain parts of the image spam messages to get past my detections. It usually takes me a few minutes to react to a new combination of give-away codes that I can block in a new or altered rule.

I am going to reduce the most common rules into one filter to save processing time, which can get a bit long. Also, I reduced my scanning in MWP to 200 lines, from 300, and that helps get spam identified more quickly.

I am trying to come up with a ruleset to detect when there is at least so many bytes of ascii characters and so many bytes of base64. All of these image spams have a combination of about 3 kb text/html and ~15kb base64 Gif. But, this would really slow down the processing time.
Author: Ikeb,  PostPosted: Mon Nov 06, 2006 8:58 pm    Post subject:
OK, I placed the link along with some explanatory text at the CastleCopsWiki's MWP Filter page. Wizcrafts, perhaps you might have some examples etc that could be added to the Header filtering or Body filtering articles. If so, you're most certainly encouraged to contribute the examples with explanation to CCW. Smile

BTW, I split the Wizcrafts filter set discussion to its own topic so it's more likely to be viewed by all. A lot of users have been asking for something like this especially given that Gary hasn't updated his filter set for several years now. On behalf of all MWP users, a big thank you to Wizcrafts!
Author: Wizcrafts, Location: Michigan PostPosted: Mon Nov 06, 2006 9:39 pm    Post subject: Editing Wiki for MWP Filters
Ikeb wrote:
OK, I placed the link along with some explanatory text at the CastleCopsWiki's MWP Filter page. Wizcrafts, perhaps you might have some examples etc that could be added to the Header filtering or Body filtering articles. If so, you're most certainly encouraged to contribute the examples with explanation to CCW. Smile

BTW, I split the Wizcrafts filter set discussion to its own topic so it's more likely to be viewed by all. A lot of users have been asking for something like this especially given that Gary hasn't updated his filter set for several years now. On behalf of all MWP users, a big thank you to Wizcrafts!


Ikeb;
I'll try adding some of my examples and logic to the filtering section of the MWP Wiki, after I analyze what's already in it.
Author: Wizcrafts, Location: Michigan PostPosted: Wed Nov 08, 2006 4:49 pm    Post subject:
If you are experiencing slowdowns in displaying or hiding email in MWP after installing my filters, it may be the fault of GIF Spam rules 3 and 4. I found that they were testing for too many broad matches in the Regular Expressions, for body text.

I just updated (on Nov 8, 2006) the GIF Spam #3 and #4 filters to speed up parsing. Grab the new code at www.wizcrafts.net/docs/filters.txt . The rest of the filter rules remain unchanged since Nov 6, 2006.

Please let me know if you continue to experience slowdowns, or if MailWasher Pro becomes unresponsive while downloading messages, after installing my filters. I'll try to find and fix the problem (usually RegExpr). So far only the image spam filters have cause me any problems in rendering.
Author: Wizcrafts, Location: Michigan PostPosted: Wed Nov 08, 2006 9:06 pm    Post subject:
Sorry guys. My last update didn't work out as planned. I have reverted to the previous matching of body text. If I find a better workaround I'll let you know. In the meantime, here are the correct rules that I tried to speed up, unsuccessfully, as the need to be, for now. Each rule should occupy one continuous line, without spaces between them.

[enabled],"GIF Spam #3","GIF Spam#3",16711680,AND,Hidden,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,containsRE,"<img\ alt="".*"" ",Body,containsRE,"src=""cid:.+@.+",Body,containsRE,"\ width="".+""><br>",Body,containsRE,"(.+<br>){10,}",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Disposition: inline;"
[enabled],"GIF Spam #4","GIF Spam#4",16711680,AND,Hidden,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,containsRE,"<img\salt="".*""",Body,containsRE,"src=""cid:.+""><br>",Body,containsRE,"(.+<br>){8,}",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64"

Paste these over the ones you replaced earlier today. Avoid extra spaces after the end of a line of code. Do not allow blank lines between rules, or after the last rule.

If anybody with knowledge of RegExpr want to help me figure this parsing speed problem send me a PM.
Author: fusion789689, Location: USA PostPosted: Sun Nov 12, 2006 2:28 pm    Post subject:
I downloaded the new filters and made the amendment to the 'gif spam #3' and '#4' rules. Installed and rebooted Mailwasher. All ran fine for a couple of hours.

After a couple of hours of recent shutdown, I have come home to Mailwasher continuously crashing. Simply locks up when trying to download one of the emails. Was a spam mail, because I have put the old rules.txt file back and all is fine. There wasn't one from anyone I know.

Point is one of the rules caused a massive hang. I waited max 3 mins before a force quit. Even if this is normal, I'm certainly not going through this again. It was only downloaded and installed today, and after a total of around 2-3 hours, a rule is causing chaos. Never got the route of which one as Mailwasher never completed the process.

Task manager was reporting 100% CPU useage. XP sp2 and Mailwasher 5.3. Shame this and having to go back to the old rules from years previous.

Thanks a lot for all the effort put into the production of these and for making them public, but nonetheless hoping for a fixed update soon!
Author: Wizcrafts, Location: Michigan PostPosted: Sun Nov 12, 2006 5:48 pm    Post subject:
fusion789689 wrote:
I downloaded the new filters and made the amendment to the 'gif spam #3' and '#4' rules. Installed and rebooted Mailwasher. All ran fine for a couple of hours.

After a couple of hours of recent shutdown, I have come home to Mailwasher continuously crashing. Simply locks up when trying to download one of the emails. Was a spam mail, because I have put the old rules.txt file back and all is fine. There wasn't one from anyone I know.

Point is one of the rules caused a massive hang. I waited max 3 mins before a force quit. Even if this is normal, I'm certainly not going through this again. It was only downloaded and installed today, and after a total of around 2-3 hours, a rule is causing chaos. Never got the route of which one as Mailwasher never completed the process.

Task manager was reporting 100% CPU useage. XP sp2 and Mailwasher 5.3. Shame this and having to go back to the old rules from years previous.

Thanks a lot for all the effort put into the production of these and for making them public, but nonetheless hoping for a fixed update soon!


Fusion;
You are correct. I have been working on the problem and at the same time the spammers are altering their codes again. I now find that there are two basic applications being employed to create these image spam messages. One of these applications is being altered every few days to try to elude learning or MWP filters, and I am on it. The following rules are catching 99% of the image spam coming in, as of this morning, without pegging Mailwasher at 100% for more than a few seconds.

Note: I also moved the slider down to 200 lines of code in the Options. None of these messages has meaningful quotable text anywhere near that figure. In fact, the average filesize of GIF Spams has decreased from about 17 kb to around the 12kb level, most of which is the base64 content of the inline gif.

These rules should each occupy only one continuous line, without extra spaces after the end characters, and no blank lines between or after a rule. Close MailWasher before adding these rules to filters.txt, or they will be overwritten when the program closes.

[enabled],"GIF Spam #4","GIF Spam#4",16711680,AND,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,containsRE,"<img\ (alt="".*"")?",Body,containsRE,"src=""cid:.+""><br>",Body,containsRE,"(.+<br>){6,}",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64"

[enabled],"GIF Spam #2","GIF Spam#2",16711680,AND,Delete,Automatic,Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64",Body,contains,_NextPart_,Body,containsRE,"<META\ http-equiv=3DContent-Type\ content=3D""text/html;.*|<meta\ content=""text/html;charset=ISO-8859-1""\ http-equiv=""Content-Type"">",Body,contains,"Content-Transfer-Encoding: quoted-printable",EntireHeader,contains,"X-Mailer: Microsoft Outlook Express 6.00.",Body,containsRE,"<IMG alt=3D"".*"" hspace=3D0",Body,containsRE,"(.+\s=){8,}",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,contains,"boundary=""----=_NextPart_"

Let me know if this helps with the CPU consumption and program lockups.

Last edited by Wizcrafts on Sun Nov 12, 2006 6:04 pm, edited 1 time in total
Author: Trapper, Location: UK PostPosted: Sun Nov 12, 2006 6:03 pm    Post subject:
From a beginners point of view, any chance of some instructions on how to put these filters into MW Pro? Do you just highlight, copy and paste 'as is' or do you need to do anything else? I don't want to end up with a dead MW Pro!!

Secondly, and maybe a bit off subject but still to do with filters, is it possible to automatically delete (So not seen) spam as noted as spam by relays and spamcop? I see that nearly all the emails I receive are listed as 'Origin Blacklisted' If these are known spams do I need to see them at all or can I set MW Pro to delete these but show me ones that it doesn't recognise so I can delete them myself?

Hope that makes sense Confused Confused
CastleCops -> Mailwasher - Troubleshooting / General

All times are GMT

Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
Page 1 of 9


Powered by phpBB © 2001 phpBB Group