| View previous topic :: View next topic |
| Author |
Message |
Jim Stephens
Guest IP: 216.78.*.*
|
Posted: Fri May 30, 2003 4:06 pm Post subject: Increasing effectiveness of spam-killing |
|
|
I'd tried it all, blacklist, filtering keywords like "viagra" and "penis" and "mortgage", blah blah blah, but the most effective thing I have done is to kill all offshore-originating e-mails. I have set up the filters to look in the entire header, and anything that contains, for example, .co.jp or .fr or .co.uk (and the list goes on and on) automatically goes bye-bye.
|
|
| Back to top |
|
 |
Al
Captain

 Joined: May 08, 2002 Posts: 331 Location: Australia
|
Posted: Fri May 30, 2003 10:38 pm Post subject: |
|
|
Could have saved yourself the trouble by downloading and installing the "prebuilt" filters available, via MWP.
See the Mailwasher - Troubleshooting / General forum for links to "Beta" filters. It's already got a filter to do this. _________________ Al
|
|
| Back to top |
|
 |
mole
Lieutenant
 Premium Member
 Joined: Mar 15, 2003 Posts: 201 Location: Afghanistan
|
Posted: Sun Jun 01, 2003 3:05 am Post subject: |
|
|
I don't use filters because they generally don't work. Spammers design emails today with beating filters in mind.
Try activating and strengthening your DNS Server blacklist. _________________ .BIZ rocks!
|
|
| Back to top |
|
 |
gary
Lieutenant
 Premium Member
 Joined: Dec 22, 2002 Posts: 260 Location: Dallas/Ft. Worth, USA
|
Posted: Sun Jun 01, 2003 5:15 am Post subject: |
|
|
Oh, I think they work to some extent, or at least they do for me. But just like anything else, they are only part of the arsenal. Every method has its advantages and disadvantages. I get a fairly good hit rate with the filters I've been using, but there are spammers who will give you a subject line that has nothing to do with the e-mail, use your address in the TO field, not use any special tricks in the HTML, put in a correct date, etc. So, combining filters with other methods, such as Bayesian analysis, blacklists, etc. seems to be the best way to go for me.
That said, if Jim is getting good results by filtering foreign top level domains, then more power to him! Besides, writing filters is educational! Why, I've learned more synonyms for male and female body parts than I EVER thought possible! _________________ Gary
|
|
| Back to top |
|
 |
Al
Captain

 Joined: May 08, 2002 Posts: 331 Location: Australia
|
Posted: Sun Jun 01, 2003 7:58 am Post subject: |
|
|
| mole wrote: | | I don't use filters because they generally don't work. Spammers design emails today with beating filters in mind. |
FWIW, it's EXTREMELY unusual for SPAM to get past the filters I use, which are basically as they come. _________________ Al
|
|
| Back to top |
|
 |
mole
Lieutenant
 Premium Member
 Joined: Mar 15, 2003 Posts: 201 Location: Afghanistan
|
Posted: Sun Jun 01, 2003 9:37 am Post subject: |
|
|
Don't get me wrong. Filters are great to pluck things out. MWP's 2 in-house filters that popped out from the beta upgrade prototypes flag tons of spam above and beyond the known spam list.
But for the sophisticated types of spam, you will need to set your filters so tight that false positives become a real issue.
For those who don't have the time to constantly tweak their filters, this can be a pain.
DNS BL is the quick and easy was out. _________________ .BIZ rocks!
|
|
| Back to top |
|
 |
Ikeb
Special Response Team Forums Admin
 Joined: Apr 20, 2003 Posts: 16509
|
Posted: Sun Jun 01, 2003 5:40 pm Post subject: |
|
|
| mole wrote: | | Don't get me wrong. Filters are great to pluck things out. MWP's 2 in-house filters that popped out from the beta upgrade prototypes flag tons of spam above and beyond the known spam list. |
Have you tried Gary's filters?
| Quote: | | But for the sophisticated types of spam, you will need to set your filters so tight that false positives become a real issue. |
True, filters have to be used with some level of caution. I find that some of Gary's filters give me 100% confidence as to whether the email is SPAM but others (e.g. ALL CAPS Subjects) pick off people who send me email in which they want to read ASAP for example. At the very least though, such filters draws attention to the "suspicious" messages. I may not even mark such flters with message deletion but at least the filters flag messages I should look at.
| Quote: | | For those who don't have the time to constantly tweak their filters, this can be a pain. |
Exactly. That's why I very much appreciate Gary's efforts.
| Quote: | | DNS BL is the quick and easy was out. |
And you say you don't like false positives?
|
|
| Back to top |
|
 |
Ikeb
Special Response Team Forums Admin
 Joined: Apr 20, 2003 Posts: 16509
|
Posted: Sun Jun 01, 2003 5:49 pm Post subject: |
|
|
| gary wrote: | | So, combining filters with other methods, such as Bayesian analysis, blacklists, etc. seems to be the best way to go for me. |
I agree. From my understanding of Bayesian filtering, it's very promising technique. While I could use K9 to apply probability filtering, it doesn't integrate well with MWP. Any chance of talking Nick into getting some Bayesian filtering added to MWP?
|
|
| Back to top |
|
 |
gary
Lieutenant
 Premium Member
 Joined: Dec 22, 2002 Posts: 260 Location: Dallas/Ft. Worth, USA
|
Posted: Sun Jun 01, 2003 5:52 pm Post subject: |
|
|
That would be nice! We should submit that as a suggestion. In the meantime, I've been using POPFile (popfile.sourceforge.net) and it's been working great. You need to modify two of the Perl files, but I can post the modifications if anyone is interested. _________________ Gary
|
|
| Back to top |
|
 |
Ikeb
Special Response Team Forums Admin
 Joined: Apr 20, 2003 Posts: 16509
|
|
| Back to top |
|
 |
gary
Lieutenant
 Premium Member
 Joined: Dec 22, 2002 Posts: 260 Location: Dallas/Ft. Worth, USA
|
Posted: Mon Jun 02, 2003 3:39 pm Post subject: |
|
|
*Der* "We should submit that as a suggestion" - I guess I forgot which forum we were in! Hehe
Ikester, I've not tried K9, but I've been meaning to take a look at it. Maybe then I can get a feeling for how the two compare. Most of those statistical filters don't seem to like the TOP command, so that's why I went with POPfile, because the code is easily modified. I'll post the instructions to the Web site pretty soon (I've been saying that for several weeks now, haven't I?). You don't need to use the cross platform version, just the Windows version. You have to dig a bit in order to get all of the info on POPFile and its theory of operation, but if you go to their forum, you'll see all sorts of interesting things. _________________ Gary
|
|
| Back to top |
|
 |
mole
Lieutenant
 Premium Member
 Joined: Mar 15, 2003 Posts: 201 Location: Afghanistan
|
Posted: Tue Jun 03, 2003 5:02 am Post subject: |
|
|
FWIW, DNS Blacklists take into account rouge IP addresses as well.
False positives are detected with the eye, so far I find that filters tag more false positives than DNS Blacklists (the good ones). But there are of course good filter sets created by people like Gary.
I guess if you really like to tweak filters, then it is a good option to have. _________________ .BIZ rocks!
|
|
| Back to top |
|
 |
|
|