CastleCops, Internet Crime Fighters
Need help? Click here to register for free! Absolutely zero advertisements on this site!

$9736.22 of $21422.68
left sidedonated so farneed $11686.46 donated to reach our goalright side, our goal
Help CastleCops serve the community on new servers, Donate Here to reach our goal.

Donation/Premium
spacer
block bottom
Security Central
spacer
· Home
· PIRT/Fried Phish
· MIRT
· SIRT
· Deutsch
· Wiki
· Newsletter
· O16/ActiveX
· CLSID List
· Contest2007
· Downloads
· Feedback (send)
· Forums
· HijackThis
· Hijacktrend
· LSPs
· My Downloads
· O18
· O20
· O21
· O22
· O23
· O9
· Premium
· Private Messages
· Proxomitron
· Reviews
· Search
· StartupList
· Stories Archive
· Submit News
· WsIRT
· Your Account
· Acceptable Use Policy
block bottom
Survey
spacer
Was 2007 a good year?

Yes it was a wonderful year
Yes, but there is always room for improvement
Status quo
It was a challenge
Other (leave comment)



Results
Polls

Votes: 941
Comments: 25
block bottom
spacer spacer

A new filter set for MWP users brought to you by Wizcrafts!
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9  Next
 
Post new topic   Reply to topic       All -> FavForums -> Mailwasher - Troubleshooting / General [del.icio.us!] [digg it!] [reddit!]
View previous topic :: View next topic  
Author Message
stan_qaz

Premium Member


Joined: Mar 31, 2003
Posts: 10615

Premium

PostPosted: Sun Nov 12, 2006 6:28 pm    Post subject:
Reply with quote

Trapper wrote:
is it possible to automatically delete (So not seen) spam as noted as spam by relays and spamcop? I see that nearly all the emails I receive are listed as 'Origin Blacklisted' If these are known spams do I need to see them at all or can I set MW Pro to delete these but show me ones that it doesn't recognise so I can delete them myself?


The origin blacklisted flag doesn't indicate the message is spam, only that it comes from a server that has a history of sending spam. This can be a very important difference when a legitimate ISP gets listed for not keeping spam off the net.

If you switch the Origin of Spam tool to delete automatically any spam not tagged as good by another tool will be automatically deleted. I'd not recommend this due to the potential for error.


_________________
Questions? Try the wiki
http://wiki.castlecops.com/MailWasher_Pro
Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Sun Nov 12, 2006 6:44 pm    Post subject:
Reply with quote

Trapper wrote:
From a beginners point of view, any chance of some instructions on how to put these filters into MW Pro? Do you just highlight, copy and paste 'as is' or do you need to do anything else? I don't want to end up with a dead MW Pro!!

Secondly, and maybe a bit off subject but still to do with filters, is it possible to automatically delete (So not seen) spam as noted as spam by relays and spamcop? I see that nearly all the emails I receive are listed as 'Origin Blacklisted' If these are known spams do I need to see them at all or can I set MW Pro to delete these but show me ones that it doesn't recognise so I can delete them myself?

Hope that makes sense Confused Confused


Trapper;
MailWasher Pro stores it's filters and blacklist/whiltelist in a sub-directory of Application Data, in your login-name directory. The simplest way to find this directory (folder) is, with MailWasher Pro open, to click on Help, then on About, then on the blue link near the bottom of the About MailWasher window-ette (it's a smll window, so it's a window-ette!). It will show a pth like this example:
C:\Documents and Settings\{Your Login Name}\Application Data\MailWasherPro\

In that folder there will be a file named filters.txt. Close out MailWasher Pro before editing this file, then refresh the MailWasherPro folder by pressing F5. If you have other custom rules you should backup the filters by opeining the filters.txt in notepad, then saving it as filters-prev.txt. Close the file and open filters.txt again, in Notepad. Turn off Word Wrap. Highlight and copy one of my rules at a time and paste it into filters.txt. I strongly recommend inserting it after the rule for email restored by MWP.

Ensure that each rule starts on a separate line, is continuous from start to finish, and that there are no spaces after the rule, or between rules, or after the final rule.

If you are going to use bulk filter rules like the set I published on my website, they are already setup correctly to be copied and pasted as is. Be sure Word Wrap is off, so you can be sure that each rule starts on it's own line. Some really long rules may spill over to a second line if you run out of character memory in Notepad, but that is ok, as long as there is no linefeed character inserted on that line. I use NoteTab Pro as my text and html editor, and it allows for incredibly long lines of code to be displayed and saved.

When you have saved filters.txt open MailWasher Pro and test the waters.

To answer your second question, any rule can be changed to Automatically Delete Without Warning, in the individual filter options. You can display your filters in the open program by clicking on View > Filter Sidebar (Control + F7). Find any rule you are certain is 100% effective, double click on it to open it for editing, then change the Action from On "Process Mail" to "Automatically (without notification)."

Blacklists and sources of spam each have their own action options. With MWP open click on the button labeled "Spam Tools" and an entire widow full of options will open. Check out each catagory and use the Options to change the action to Automatic. Be aware that unless you turn on the Recycle Bin in the program, any mistakes will be permanent. If the Restore feature is turned on and you discover a legitimate email was deleted (Control + S opens the activity log file), you can highlight that message then click Restore. You must have inputted a valid email address that allows you to send email for this to work.

I hope this helps clarify the fine points of using MWP.

When you have pasted in all applicable rules save the file,


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Trapper

Trooper
Trooper


Joined: Feb 13, 2004
Posts: 28
Location: UK

PostPosted: Sun Nov 12, 2006 7:58 pm    Post subject:
Reply with quote

Thank you for enlightening me, I was obviously confused as I thought the origin blacklisted came from the spamcop or relays black list when checked, I also assumed the senders name/domain was being checked not the senders server. I take it that not all emails from that server would be spam, hence the recommendation not to delete them automatically. Idea Smile

Many thanks WIzcraft for taking the time to explain what I should do in detail, I will print it out and read at work tomorrow Wink I should be able to get the hang of it in time Exclamation

Trapper

Back to top
View users profile Send private message
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16509

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Mon Nov 13, 2006 7:09 am    Post subject:
Reply with quote

Wizcrafts wrote:
If anybody with knowledge of RegExpr want to help me figure this parsing speed problem send me a PM.

Please just post here so that we can all learn. Smile

BTW, are you keeping the file updated at the link you supplied?

There are some filter.txt editing instructions at MailWasher_Pro_Spam_Tools_Filters#Filter_Development_Background. I just edited the page so that the instructions are highlighted. Please have a look and feel free to edit to clarify any of the current instructions.

Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Mon Nov 13, 2006 3:21 pm    Post subject:
Reply with quote

wrote:
Wizcrafts wrote:
If anybody with knowledge of RegExpr want to help me figure this parsing speed problem send me a PM.

Please just post here so that we can all learn. Smile

BTW, are you keeping the file updated at the link you supplied?

There are some filter.txt editing instructions at MailWasher_Pro_Spam_Tools_Filters#Filter_Development_Background. I just edited the page so that the instructions are highlighted. Please have a look and feel free to edit to clarify any of the current instructions.
Ikeb;
I am keeping it updated. I have adjusted the expressions two or three times since posting about it here. I'm also trying to reduce the load it imposes on MWP, by reducing the expressions to be matched.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16509

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Mon Nov 13, 2006 6:55 pm    Post subject:
Reply with quote

That's great!

Re: reducing load, it sure would be great to have some built-in diagnostic method to pinpoint and troubleshoot poorly-performing filters.

Back to top
View users profile Send private message
fusion789689

Cadet
Cadet


Joined: Nov 12, 2006
Posts: 3
Location: USA

PostPosted: Sat Nov 18, 2006 6:20 am    Post subject:
Reply with quote

Wizcrafts wrote:

Let me know if this helps with the CPU consumption and program lockups.


Thanks a lot for further amendments and the instructions on how to tweak what MWP should be obtaining from each mail to display in the app itself. I'll install it again new with new rules you posted and see what happens. However, if these spammers are modifying code often, surely this scenario might crop up again. I do agree with IKeb's comment: 'Re: reducing load, it sure would be great to have some built-in diagnostic method to pinpoint and troubleshoot poorly-performing filters.' This would obviously be most advantageous: to find those that tend to lock the app up. More from me later with feedback... I'll try run these for a couple of days, unless I have no luck sooner!

Back to top
View users profile Send private message
fusion789689

Cadet
Cadet


Joined: Nov 12, 2006
Posts: 3
Location: USA

PostPosted: Sat Nov 18, 2006 6:24 am    Post subject:
Reply with quote

Heck, maybe you need to ignore my comments... I do see you mentioned (after my post) about the 'word wrap' option.... I didn't switch this 'off' when pasting all rules and new GIF amendments into the wordpad filters.txt doc... so this might have caused the lock-ups...! Will give it all a bash now after further enlightenment. Smile

Back to top
View users profile Send private message
Trapper

Trooper
Trooper


Joined: Feb 13, 2004
Posts: 28
Location: UK

PostPosted: Sat Nov 18, 2006 7:34 pm    Post subject:
Reply with quote

Well, I have copied all the rules 'as is' and with MWP closed added them to my original rules set. I called my original set 'filters old' and the new set 'filters'.

On restarting MWP, it began to show 2 of 31 and then froze.

I shut down and got the same 2 of 31.
I have reinstalled my original 'filter old' and all seems okay Smile

I see there are some alterations to your rules as per the post of fusion, now do you need to find the original rule, delete it and replace with these new versions or will these new versions overwrite the original rules Question Sorry for being so ignorant Exclamation

Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Sat Nov 18, 2006 8:17 pm    Post subject:
Reply with quote

Trapper;
I have been trying to correct the resource overload caused by the image spam filters. I also noticed that the spammers are altering their codes considerably. Only one of my original gif spam rules is still effective, as it was. The rest have morphed into variations upon variations.

My filter rules depend on matching some regular expressions, and that is where the freeze ups are occuring. The processing power requirements are quite intense when looking at 200 lines of code, especially when you encounter base64 codes.

I am working on this everyday and will come up with a good filter for gif spam, that does not grind the program to a halt.

In the meantime, I have changed the order of the useful filters, as they pertain to the current crop of these spam messages. The following four filters are now catching most of the image spam this week...

[enabled],"GIF Spam #4c","GIF Spam#4b",16711680,AND,Hidden,Delete,Automatic,EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,containsRE,"boundary="".+""",Body,contains,"Content-Transfer-Encoding: 7bit",Body,contains,"<img ",Body,containsRE,"src=""cid:.+""><br>",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64",Body,doesn'tContain,"<a href="

[enabled],"Image Spam Type B","Image Spam Type B",16711680,AND,Hidden,Delete,Body,contains,"</style><img src=",Body,contains,<style>

[enabled],"OE GIF SPAM #2","OE GIF Spam#2",16711680,AND,Hidden,Delete,EntireHeader,contains,"X-Mailer: Microsoft Outlook Express 6.00.",EntireHeader,contains,"X-MimeOLE: Produced By Microsoft MimeOLE V6.00.",EntireHeader,contains,"MIME-Version: 1.0",EntireHeader,contains,"Content-Type: multipart/mixed;",EntireHeader,contains,"boundary=""----=_NextPart_",Body,contains,"Content-Type: multipart/alternative;",Body,contains,"Content-Transfer-Encoding: quoted-printable",Body,contains,"Content-Transfer-Encoding: base64",Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Disposition: attachment;"

[enabled],"OE GIF Spam #1","OE Gif Spam#1",16711680,AND,Hidden,Delete,Automatic,Body,contains,"Content-Type: image/gif;",Body,contains,"Content-Transfer-Encoding: base64",Body,contains,_NextPart_,Body,containsRE,"<META\ http-equiv=3DContent-Type\ content=3D""text/html;.*|<meta\ content=""text/html;charset=ISO-8859-1""\ http-equiv=""Content-Type"">",Body,contains,"Content-Transfer-Encoding: quoted-printable",EntireHeader,contains,"X-Mailer: Microsoft Outlook Express 6.00.",Body,containsRE,"<IMG alt=3D"".*"" hspace=3D0",Body,containsRE,"(.+\s=){6,}",EntireHeader,contains,"Content-Type: multipart/related;",EntireHeader,contains,"boundary=""----=_NextPart_"

I have separated the rules for clarity. Turn off word wrap to insert them into your filters.txt. As stated before, each rule needs to occupy one long line of code. There should not be any trailing blank spaces after a rule, nor blank lines between rules, and the last rule should end at the end of it's line.

I will keep at it until I succeed at catching all of the image spam without overloading MWP. Thanks for your tolerance with my experiments. I dislike spam as much as you do.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Tadiew

Trooper
Trooper


Joined: Jun 17, 2005
Posts: 14
Location: San Francisco, CA

PostPosted: Tue Nov 21, 2006 2:29 am    Post subject: Virus possibility linked to list?
Reply with quote

[quote="Wizcrafts"]Oh well, no reply, so here is a link to my custom [url=http://www.wizcrafts.net/docs/filters.txt]MWP spam filter rules[/url].</quote>

<snip>

I'm probably somehow responsible for the following (lack of info, etc.); I'd imagine otherwise someone else would have commented on it some time ago here.

I visited your filters page earlier today to see if I could find anything regarding a filters question I have (and will post after this) and McAfee (VirusScan v.11) threw up a warning that it had detected and deleted the Exploit-MIME.gen virus in my browser's cache (Firefox 2.0). I'm on Windows 2000 Pro.

I was out for several hours after that and when I went back just now to get the entire warning the page was no longer available.

I hope all's well on your end and that my concern was totally unnecessary.

Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Tue Nov 21, 2006 3:16 am    Post subject:
Reply with quote

Tadiew;
My filters are in a plain ASCII text file. There are no exploits possible with a .txt extension, to my knowledge. I can't even make a clickable hyperlink out of a url in the file. I just checked it and it is as I last uploaded it this morning.

I'm not the cause of your virus warning, but I remember back in the days of Thunderbyte anti virus that it shipped with a text file called eicar.txt that was used to test the program. Apparently, certain text patterns can trigger a false positive in heuristic anti virus engines. Maybe my filter rules are like that old eicar test file. At least you know your anti virus program has a heuristic engine, in addition to the definitions engine.

The coding has changed again on the other side of the fence. They are now resorting to embedding images as background files, in the body tag, and to using style tags to hide stuff. It took me two minutes to detect this and another five to create a new rule to kill it. Filters.txt is now updated to detect background image spam, and has new filter names, for the image rules. I have also reduced the amount of regular expressions in the various image-spam rules, speeding up load times for the rules and shortening the scanning time considerably.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Tue Nov 21, 2006 3:25 am    Post subject:
Reply with quote

Based on the traces shown when I report some of these image spams to SpamCop, all appear to be coming from compromised computers that are in a botnet. Some are on RoadRunner, in the US, others are in Turkey, China, Malaysia, Romania, Poland and Canada.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Tadiew

Trooper
Trooper


Joined: Jun 17, 2005
Posts: 14
Location: San Francisco, CA

PostPosted: Tue Nov 21, 2006 5:22 am    Post subject: Heuristics
Reply with quote

Thank you! That makes sense. McAfee has a toggle for heuristic detection. It's interesting to see what kind of thing fires that up, lol. At least I now know it's "on the job."

I have been using Mailwasher Pro very crudely because I've had trouble figuring out how to use regular expressions properly. For a variety of reasons I'm going to put some time into trying to learn how to use them to create better filters so I'm really excited to see really good ones you have made.

Thanks and thank you.

Back to top
View users profile Send private message
Wizcrafts

Sergeant
Sergeant
Premium Member

Joined: Jun 05, 2003
Posts: 95
Location: Michigan
Premium

PostPosted: Tue Nov 21, 2006 2:54 pm    Post subject:
Reply with quote

I'll be on the road for a couple of days, until Thursday or Friday, and will check in here at that time. Please note that the image spammers are altering their codes to try to elude these filters, and the you may need to tweak a line of code, here and there.

I'll post current filters after my trip.


_________________
Submitted by Wiz
Guarding the Castle against spammers and scammers
Back to top
View users profile Send private message Visit posters website AIM Address Yahoo Messenger
Display posts from previous:   
Post new topic   Reply to topic       All -> FavForums -> Mailwasher - Troubleshooting / General All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9  Next
Page 2 of 9

 
Quick Reply:
Username: 

Quote the last message
Attach signature (signatures can be changed in profile)
 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001 phpBB Group
spacer spacer