CastleCops, Internet Crime Fighters
Need help? Click here to register for free! Absolutely zero advertisements on this site!

Donation/Premium
spacer
block bottom
Security Central
spacer
· Home
· PIRT/Fried Phish
· MIRT
· SIRT
· Deutsch
· Wiki
· Newsletter
· O16/ActiveX
· CLSID List
· Contest2007
· Downloads
· Feedback (send)
· Forums
· HijackThis
· Hijacktrend
· LSPs
· My Downloads
· O18
· O20
· O21
· O22
· O23
· O9
· Premium
· Private Messages
· Proxomitron
· Reviews
· Search
· StartupList
· Stories Archive
· Submit News
· WsIRT
· Your Account
· Acceptable Use Policy
block bottom
spacer spacer

Increasing effectiveness of spam-killing

 
Post new topic   Reply to topic       All -> FavForums -> Product Suggestions [del.icio.us!] [digg it!] [reddit!]
View previous topic :: View next topic  
Author Message
Jim Stephens

Guest
IP: 216.78.*.*






PostPosted: Fri May 30, 2003 4:06 pm    Post subject: Increasing effectiveness of spam-killing
Reply with quote

I'd tried it all, blacklist, filtering keywords like "viagra" and "penis" and "mortgage", blah blah blah, but the most effective thing I have done is to kill all offshore-originating e-mails. I have set up the filters to look in the entire header, and anything that contains, for example, .co.jp or .fr or .co.uk (and the list goes on and on) automatically goes bye-bye.

Back to top
Al

Captain
Captain


Joined: May 08, 2002
Posts: 331
Location: Australia

PostPosted: Fri May 30, 2003 10:38 pm    Post subject:
Reply with quote

Could have saved yourself the trouble by downloading and installing the "prebuilt" filters available, via MWP.

See the Mailwasher - Troubleshooting / General forum for links to "Beta" filters. It's already got a filter to do this.


_________________
Al
Back to top
View users profile Send private message Send email Visit posters website MSN Messenger
mole

Lieutenant
Lieutenant
Premium Member

Joined: Mar 15, 2003
Posts: 201
Location: Afghanistan
Premium

PostPosted: Sun Jun 01, 2003 3:05 am    Post subject:
Reply with quote

I don't use filters because they generally don't work. Spammers design emails today with beating filters in mind.

Try activating and strengthening your DNS Server blacklist.


_________________
.BIZ rocks!
Back to top
View users profile Send private message Visit posters website
gary

Lieutenant
Lieutenant
Premium Member

Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
Premium

PostPosted: Sun Jun 01, 2003 5:15 am    Post subject:
Reply with quote

Oh, I think they work to some extent, or at least they do for me. But just like anything else, they are only part of the arsenal. Every method has its advantages and disadvantages. I get a fairly good hit rate with the filters I've been using, but there are spammers who will give you a subject line that has nothing to do with the e-mail, use your address in the TO field, not use any special tricks in the HTML, put in a correct date, etc. So, combining filters with other methods, such as Bayesian analysis, blacklists, etc. seems to be the best way to go for me.

That said, if Jim is getting good results by filtering foreign top level domains, then more power to him! Besides, writing filters is educational! Why, I've learned more synonyms for male and female body parts than I EVER thought possible!


_________________
Gary
Back to top
View users profile Send private message
Al

Captain
Captain


Joined: May 08, 2002
Posts: 331
Location: Australia

PostPosted: Sun Jun 01, 2003 7:58 am    Post subject:
Reply with quote

mole wrote:
I don't use filters because they generally don't work. Spammers design emails today with beating filters in mind.


FWIW, it's EXTREMELY unusual for SPAM to get past the filters I use, which are basically as they come.


_________________
Al
Back to top
View users profile Send private message Send email Visit posters website MSN Messenger
mole

Lieutenant
Lieutenant
Premium Member

Joined: Mar 15, 2003
Posts: 201
Location: Afghanistan
Premium

PostPosted: Sun Jun 01, 2003 9:37 am    Post subject:
Reply with quote

Don't get me wrong. Filters are great to pluck things out. MWP's 2 in-house filters that popped out from the beta upgrade prototypes flag tons of spam above and beyond the known spam list.

But for the sophisticated types of spam, you will need to set your filters so tight that false positives become a real issue.

For those who don't have the time to constantly tweak their filters, this can be a pain.

DNS BL is the quick and easy was out.


_________________
.BIZ rocks!
Back to top
View users profile Send private message Visit posters website
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16534

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Sun Jun 01, 2003 5:40 pm    Post subject:
Reply with quote

mole wrote:
Don't get me wrong. Filters are great to pluck things out. MWP's 2 in-house filters that popped out from the beta upgrade prototypes flag tons of spam above and beyond the known spam list.

Have you tried Gary's filters?
Quote:
But for the sophisticated types of spam, you will need to set your filters so tight that false positives become a real issue.

True, filters have to be used with some level of caution. I find that some of Gary's filters give me 100% confidence as to whether the email is SPAM but others (e.g. ALL CAPS Subjects) pick off people who send me email in which they want to read ASAP for example. At the very least though, such filters draws attention to the "suspicious" messages. I may not even mark such flters with message deletion but at least the filters flag messages I should look at.
Quote:
For those who don't have the time to constantly tweak their filters, this can be a pain.

Exactly. That's why I very much appreciate Gary's efforts.

Quote:
DNS BL is the quick and easy was out.

And you say you don't like false positives? Razz

Back to top
View users profile Send private message
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16534

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Sun Jun 01, 2003 5:49 pm    Post subject:
Reply with quote

gary wrote:
So, combining filters with other methods, such as Bayesian analysis, blacklists, etc. seems to be the best way to go for me.

I agree. From my understanding of Bayesian filtering, it's very promising technique. While I could use K9 to apply probability filtering, it doesn't integrate well with MWP. Any chance of talking Nick into getting some Bayesian filtering added to MWP?

Back to top
View users profile Send private message
gary

Lieutenant
Lieutenant
Premium Member

Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
Premium

PostPosted: Sun Jun 01, 2003 5:52 pm    Post subject:
Reply with quote

That would be nice! We should submit that as a suggestion. In the meantime, I've been using POPFile (popfile.sourceforge.net) and it's been working great. You need to modify two of the Perl files, but I can post the modifications if anyone is interested.


_________________
Gary
Back to top
View users profile Send private message
Ikeb

Special Response Team
Forums Admin

Joined: Apr 20, 2003
Posts: 16534

Forums Admin Moderators MVP Premium SRT Team CC Committee Team F@H

PostPosted: Mon Jun 02, 2003 5:07 am    Post subject:
Reply with quote

gary wrote:
That would be nice! We should submit that as a suggestion.

Bayesian filtering has already been suggested but Firetrust seems more interested in offering CFS as the way of the future. Having a trusted relationship with FireTrust, I was kinda hoping you could bend Nick's ear a little. Wink
Quote:
In the meantime, I've been using POPFile (popfile.sourceforge.net) and it's been working great. You need to modify two of the Perl files, but I can post the modifications if anyone is interested.

Colour me interested! Very Happy

While you're at it would you mind publishing how you interwork POPFile, MWP, and the email client? A logical and config view would be helpful.

Suggestion: Rather than posting here, put the info at your web site for easier future reference. Yes

BTW, by having to modify the Perl source files does this mean I would use the cross-platform version?

Also, I noted that the POPFile site makes no mention of Bayesian filtering nor does it discuss the theory of POPFile operation. Do you have any experience as to how POPFile performance compares with K9 for example?

Back to top
View users profile Send private message
gary

Lieutenant
Lieutenant
Premium Member

Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
Premium

PostPosted: Mon Jun 02, 2003 3:39 pm    Post subject:
Reply with quote

*Der* "We should submit that as a suggestion" - I guess I forgot which forum we were in! Hehe

Ikester, I've not tried K9, but I've been meaning to take a look at it. Maybe then I can get a feeling for how the two compare. Most of those statistical filters don't seem to like the TOP command, so that's why I went with POPfile, because the code is easily modified. I'll post the instructions to the Web site pretty soon (I've been saying that for several weeks now, haven't I?). You don't need to use the cross platform version, just the Windows version. You have to dig a bit in order to get all of the info on POPFile and its theory of operation, but if you go to their forum, you'll see all sorts of interesting things.


_________________
Gary
Back to top
View users profile Send private message
mole

Lieutenant
Lieutenant
Premium Member

Joined: Mar 15, 2003
Posts: 201
Location: Afghanistan
Premium

PostPosted: Tue Jun 03, 2003 5:02 am    Post subject:
Reply with quote

FWIW, DNS Blacklists take into account rouge IP addresses as well.

False positives are detected with the eye, so far I find that filters tag more false positives than DNS Blacklists (the good ones). But there are of course good filter sets created by people like Gary.

I guess if you really like to tweak filters, then it is a good option to have.


_________________
.BIZ rocks!
Back to top
View users profile Send private message Visit posters website
Display posts from previous:   
Post new topic   Reply to topic       All -> FavForums -> Product Suggestions All times are GMT
Page 1 of 1

 
Quick Reply:
Username: 

Quote the last message
Attach signature (signatures can be changed in profile)
 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001 phpBB Group
spacer spacer