CastleCops, Internet Crime Fighters
Need help? Click here to register for free! Absolutely zero advertisements on this site!

$9736.22 of $21422.68
left sidedonated so farneed $11686.46 donated to reach our goalright side, our goal
Help CastleCops serve the community on new servers, Donate Here to reach our goal.

Donation/Premium
spacer
block bottom
Security Central
spacer
· Home
· PIRT/Fried Phish
· MIRT
· SIRT
· Deutsch
· Wiki
· Newsletter
· O16/ActiveX
· CLSID List
· Contest2007
· Downloads
· Feedback (send)
· Forums
· HijackThis
· Hijacktrend
· LSPs
· My Downloads
· O18
· O20
· O21
· O22
· O23
· O9
· Premium
· Private Messages
· Proxomitron
· Reviews
· Search
· StartupList
· Stories Archive
· Submit News
· WsIRT
· Your Account
· Acceptable Use Policy
block bottom
Survey
spacer
Was 2007 a good year?

Yes it was a wonderful year
Yes, but there is always room for improvement
Status quo
It was a challenge
Other (leave comment)



Results
Polls

Votes: 927
Comments: 25
block bottom
spacer spacer

A new filter set for MWP users brought to you by Wizcrafts!
Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
 
Post new topic   Reply to topic       All -> FavForums -> Mailwasher - Troubleshooting / General [del.icio.us!] [digg it!] [reddit!]
View previous topic :: View next topic  
Author Message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Sun Nov 05, 2006 3:50 pm    Post subject:
Reply with quote

Ikeb;
Am I allowed to post a link to the image filters on my own website, or should I copy and paste them into a forum reply?


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Sun Nov 05, 2006 6:22 pm    Post subject:
Reply with quote

Oh well, no reply, so here is a link to my custom MWP spam filter rules. The 2nd through 4th rules deal with the current crop of image spam for stocks and investments. Note that I have set them to automatically delete without notice. You may want to change that to "prompt," until you are certain that the rules are 100% accurate for your use.

Since the spammers occasionally alter their headers I alter my rules to match their techniques, usually within mere minutes of finding a new trick that gets past an existing filter. Such is the case with the GIF Spam #3 rule. However, even though they alter the layout there are always some things that must remain constant for their spam to work, and I always find those commonalities and create rules to detect it. I have a topic on my blog about this image spam issue and it gets a lot of hits.

I hope this helps other MailWasher Pro users. If the links are a problem I apologize in advance.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
stan_qaz

Premium Member


Joined: Mar 31, 2003
Posts: 10513

Premium

PostPosted: Sun Nov 05, 2006 8:52 pm    Post subject:
Reply with quote

Interesting collection of filters, Thanks for posting them.


_________________
Questions? Try the wiki
http://wiki.castlecops.com/MailWasher_Pro
Back to top
View users profile Send private message
BuckeyeBasser

SRT Trainee
SRT Trainee
Premium Member

Joined: May 01, 2006
Posts: 692
Location: Central Ohio, USA
Premium Team F@H

PostPosted: Sun Nov 05, 2006 8:54 pm    Post subject:
Reply with quote

Be sure you close MailWasher completely before manually editing your filter file.

Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Sun Nov 05, 2006 10:00 pm    Post subject:
Reply with quote

stan_qaz wrote:
Interesting collection of filters, Thanks for posting them.

You're welcome Stan, and other members! I'll post back here, or start a new thread, if the image spam coding vector changes and I create new rules to deal with it.

BTW: If you edit your own filters, there is a function test button for Regular Expressions. I use it against samples taken from the source code of actual spam messages to fine tune my filters. This is a great feature of the current flavor of MailWasher Pro.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Mon Nov 06, 2006 6:24 am    Post subject:
Reply with quote

I just updated my MailWasher filters to intercept a new variation of the investments image spam. The new filter is GIF Spam #4. grab them at: www.wizcrafts.net/docs/filters.txt . Be sure to close MailWasher before editing filters.txt. Watch out for blank lines between or after rules, or extra spaces after the end of a rule. Either will cause you problems.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16431

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Mon Nov 06, 2006 6:28 pm    Post subject:
Reply with quote

Wizcrafts wrote:
stan_qaz wrote:
Interesting collection of filters, Thanks for posting them.

You're welcome Stan, and other members! I'll post back here, or start a new thread, if the image spam coding vector changes and I create new rules to deal with it.

Sorry for the delay. I was busy elsewhere yesterday. Yeah I second Stan's sentiments, an interesting collection. Thank You I'd like to have it linked at the wiki if you don't mind -- perhaps featured as more current than Gary's filter collection? BTW, any connection to Gary's? Also, do any filters have to be tweaked to account for individual email addresses, etc.?

Wizcrafts wrote:
BTW: If you edit your own filters, there is a function test button for Regular Expressions. I use it against samples taken from the source code of actual spam messages to fine tune my filters. This is a great feature of the current flavor of MailWasher Pro.

Very limited though. It's much better to use Sorokin's TRegExpr when developing a filter.


_________________
imageCastleCopsWiki
Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Mon Nov 06, 2006 7:42 pm    Post subject:
Reply with quote

Ikeb wrote:

Quote:

I'd like to have it linked at the wiki if you don't mind -- perhaps featured as more current than Gary's filter collection? BTW, any connection to Gary's? Also, do any filters have to be tweaked to account for individual email addresses, etc.?

Thanks for the good words!

I would be ecstatic to have my filters linked to in the Wiki! I have been refining my filters for several years and try to keep them mostly current. There are several rules that can probably be discarded, namely the Spam Domains #1 through #9 rules. Most of the domains listed are probably long gone by now.

I do believe that I have purged any specific personal email accounts from the list, and what remains does not require your email address to function. There are already plenty of examples about using custom filters that account for your own email addresses. I use several of those, but the rules I put on my website are much more effective for general spam, especially when BCC'd.

I don't have any connection with Gary. I appreciate his work on the filters a few years ago. I used them as the basis for my rules.

Thanks for the link to the RegExpr tester.

As I mentioned earlier, the spammers must be reading my posts because they are altering certain parts of the image spam messages to get past my detections. It usually takes me a few minutes to react to a new combination of give-away codes that I can block in a new or altered rule.

I am going to reduce the most common rules into one filter to save processing time, which can get a bit long. Also, I reduced my scanning in MWP to 200 lines, from 300, and that helps get spam identified more quickly.

I am trying to come up with a ruleset to detect when there is at least so many bytes of ascii characters and so many bytes of base64. All of these image spams have a combination of about 3 kb text/html and ~15kb base64 Gif. But, this would really slow down the processing time.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16431

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Mon Nov 06, 2006 8:58 pm    Post subject:
Reply with quote

OK, I placed the link along with some explanatory text at the CastleCopsWiki's MWP Filter page. Wizcrafts, perhaps you might have some examples etc that could be added to the Header filtering or Body filtering articles. If so, you're most certainly encouraged to contribute the examples with explanation to CCW. Smile

BTW, I split the Wizcrafts filter set discussion to its own topic so it's more likely to be viewed by all. A lot of users have been asking for something like this especially given that Gary hasn't updated his filter set for several years now. On behalf of all MWP users, a big thank you to Wizcrafts!


_________________
imageCastleCopsWiki
Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Mon Nov 06, 2006 9:39 pm    Post subject: Editing Wiki for MWP Filters
Reply with quote

Ikeb wrote:
OK, I placed the link along with some explanatory text at the CastleCopsWiki's MWP Filter page. Wizcrafts, perhaps you might have some examples etc that could be added to the Header filtering or Body filtering articles. If so, you're most certainly encouraged to contribute the examples with explanation to CCW. Smile

BTW, I split the Wizcrafts filter set discussion to its own topic so it's more likely to be viewed by all. A lot of users have been asking for something like this especially given that Gary hasn't updated his filter set for several years now. On behalf of all MWP users, a big thank you to Wizcrafts!


Ikeb;
I'll try adding some of my examples and logic to the filtering section of the MWP Wiki, after I analyze what's already in it.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Wed Nov 08, 2006 4:49 pm    Post subject:
Reply with quote

If you are experiencing slowdowns in displaying or hiding email in MWP after installing my filters, it may be the fault of GIF Spam rules 3 and 4. I found that they were testing for too many broad matches in the Regular Expressions, for body text.

I just updated (on Nov 8, 2006) the GIF Spam #3 and #4 filters to speed up parsing. Grab the new code at www.wizcrafts.net/docs/filters.txt . The rest of the filter rules remain unchanged since Nov 6, 2006.

Please let me know if you continue to experience slowdowns, or if MailWasher Pro becomes unresponsive while downloading messages, after installing my filters. I'll try to find and fix the problem (usually RegExpr). So far only the image spam filters have cause me any problems in rendering.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Wed Nov 08, 2006 9:06 pm    Post subject:
Reply with quote

Sorry guys. My last update didn't work out as planned. I have reverted to the previous matching of body text. If I find a better workaround I'll let you know. In the meantime, here are the correct rules that I tried to speed up, unsuccessfully, as the need to be, for now. Each rule should occupy one continuous line, without spaces between them.

[enabled],"GIF Spam #3","GIF Spam#3",16711680,AND,Hidden,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,containsRE,"<img\ alt="".*"" ",Body,containsRE,"src=""cid:.+@.+",Body,containsRE,"\ width="".+""><br>",Body,containsRE,"(.+<br>){10,}",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Disposition: inline;"
[enabled],"GIF Spam #4","GIF Spam#4",16711680,AND,Hidden,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,containsRE,"<img\salt="".*""",Body,containsRE,"src=""cid:.+""><br>",Body,containsRE,"(.+<br>){8,}",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64"

Paste these over the ones you replaced earlier today. Avoid extra spaces after the end of a line of code. Do not allow blank lines between rules, or after the last rule.

If anybody with knowledge of RegExpr want to help me figure this parsing speed problem send me a PM.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
fusion789689

Cadet
Cadet


Joined: Nov 12, 2006
Posts: 3
Location: USA

PostPosted: Sun Nov 12, 2006 2:28 pm    Post subject:
Reply with quote

I downloaded the new filters and made the amendment to the 'gif spam #3' and '#4' rules. Installed and rebooted Mailwasher. All ran fine for a couple of hours.

After a couple of hours of recent shutdown, I have come home to Mailwasher continuously crashing. Simply locks up when trying to download one of the emails. Was a spam mail, because I have put the old rules.txt file back and all is fine. There wasn't one from anyone I know.

Point is one of the rules caused a massive hang. I waited max 3 mins before a force quit. Even if this is normal, I'm certainly not going through this again. It was only downloaded and installed today, and after a total of around 2-3 hours, a rule is causing chaos. Never got the route of which one as Mailwasher never completed the process.

Task manager was reporting 100% CPU useage. XP sp2 and Mailwasher 5.3. Shame this and having to go back to the old rules from years previous.

Thanks a lot for all the effort put into the production of these and for making them public, but nonetheless hoping for a fixed update soon!

Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Sun Nov 12, 2006 5:48 pm    Post subject:
Reply with quote

fusion789689 wrote:
I downloaded the new filters and made the amendment to the 'gif spam #3' and '#4' rules. Installed and rebooted Mailwasher. All ran fine for a couple of hours.

After a couple of hours of recent shutdown, I have come home to Mailwasher continuously crashing. Simply locks up when trying to download one of the emails. Was a spam mail, because I have put the old rules.txt file back and all is fine. There wasn't one from anyone I know.

Point is one of the rules caused a massive hang. I waited max 3 mins before a force quit. Even if this is normal, I'm certainly not going through this again. It was only downloaded and installed today, and after a total of around 2-3 hours, a rule is causing chaos. Never got the route of which one as Mailwasher never completed the process.

Task manager was reporting 100% CPU useage. XP sp2 and Mailwasher 5.3. Shame this and having to go back to the old rules from years previous.

Thanks a lot for all the effort put into the production of these and for making them public, but nonetheless hoping for a fixed update soon!


Fusion;
You are correct. I have been working on the problem and at the same time the spammers are altering their codes again. I now find that there are two basic applications being employed to create these image spam messages. One of these applications is being altered every few days to try to elude learning or MWP filters, and I am on it. The following rules are catching 99% of the image spam coming in, as of this morning, without pegging Mailwasher at 100% for more than a few seconds.

Note: I also moved the slider down to 200 lines of code in the Options. None of these messages has meaningful quotable text anywhere near that figure. In fact, the average filesize of GIF Spams has decreased from about 17 kb to around the 12kb level, most of which is the base64 content of the inline gif.

These rules should each occupy only one continuous line, without extra spaces after the end characters, and no blank lines between or after a rule. Close MailWasher before adding these rules to filters.txt, or they will be overwritten when the program closes.

[enabled],"GIF Spam #4","GIF Spam#4",16711680,AND,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,containsRE,"<img\ (alt="".*"")?",Body,containsRE,"src=""cid:.+""><br>",Body,containsRE,"(.+<br>){6,}",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64"

[enabled],"GIF Spam #2","GIF Spam#2",16711680,AND,Delete,Automatic,Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64",Body,contains,_NextPart_,Body,containsRE,"<META\ http-equiv=3DContent-Type\ content=3D""text/html;.*|<meta\ content=""text/html;charset=ISO-8859-1""\ http-equiv=""Content-Type"">",Body,contains,"Content-Transfer-Encoding: quoted-printable",EntireHeader,contains,"X-Mailer: Microsoft Outlook Express 6.00.",Body,containsRE,"<IMG alt=3D"".*"" hspace=3D0",Body,containsRE,"(.+\s=){8,}",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,contains,"boundary=""----=_NextPart_"

Let me know if this helps with the CPU consumption and program lockups.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers


Last edited by Wizcrafts on Sun Nov 12, 2006 6:04 pm, edited 1 time in total
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Trapper

Trooper
Trooper


Joined: Feb 13, 2004
Posts: 28
Location: UK

PostPosted: Sun Nov 12, 2006 6:03 pm    Post subject:
Reply with quote

From a beginners point of view, any chance of some instructions on how to put these filters into MW Pro? Do you just highlight, copy and paste 'as is' or do you need to do anything else? I don't want to end up with a dead MW Pro!!

Secondly, and maybe a bit off subject but still to do with filters, is it possible to automatically delete (So not seen) spam as noted as spam by relays and spamcop? I see that nearly all the emails I receive are listed as 'Origin Blacklisted' If these are known spams do I need to see them at all or can I set MW Pro to delete these but show me ones that it doesn't recognise so I can delete them myself?

Hope that makes sense Confused Confused

Back to top
View users profile Send private message
Display posts from previous:   
Post new topic   Reply to topic       All -> FavForums -> Mailwasher - Troubleshooting / General All times are GMT
Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
Page 1 of 9

 
Quick Reply:
Username: 

Quote the last message
Attach signature (signatures can be changed in profile)
 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001 phpBB Group
spacer spacer