|
Donation/Premium |
|
 |
|
|
|
|
|
|
|
|
|
| View previous topic :: View next topic |
| Author |
Message |
gary
Lieutenant
Premium Member
Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
|
 Posted: Tue Jun 17, 2003 1:29 am Post subject: |
|
|
If your host address is "localhost" and your e-mail address is <mail server>:<login>, I don't know what else it could be - with one exception. Are you SURE that your link target looks like:
".....\perl.exe" popfile.pl -pop3_toptoo 1
You MUST have that, otherwise POPfile will ignore MW's TOP commands.
If you view the message in raw mode, is the header or subject modification there from POPFile?
_________________
Gary
|
|
| Back to top |
|
 |
Guest
Guest
IP: 12.235.*.*
|
| Posted: Tue Jun 17, 2003 2:23 am Post subject: |
|
|
Well I did have the command line correct but I was using wperl.exe instead of perl.exe. I don't know if that matters - I tried perl.exe and no difference. I notice that when I used perl.exe I could see that PF was loading but I got an error saying that -pop3_toptoo 1 is an unknown command line option. When I do look at raw headers PF tags are not present.
|
|
| Back to top |
|
|
gary
Lieutenant
Premium Member
Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
|
| Posted: Tue Jun 17, 2003 2:32 am Post subject: |
|
|
Okay, that's the problem then.
What version of POPFile are you using? If it's older than 0.19, you need to use "-toptoo 1" as the option. It was changed to "-pop3_toptoo" in the last release, which was just a few days ago.
_________________
Gary
|
|
| Back to top |
|
|
Guest
Guest
IP: 12.235.*.*
|
| Posted: Tue Jun 17, 2003 2:48 am Post subject: |
|
|
Gary,
Yep - that did it. I just upgraded to the latest version and it all works! Thanks for all the help and staying with it - much appreciated.
|
|
| Back to top |
|
|
IP: 217.136.*.*
Guest
|
| Posted: Tue Jun 17, 2003 10:30 am Post subject: |
|
|
Gary,
As you seem to be quite up to date on both MWP and PF, I noticed that not all mails that show up in MWP show up in the PF (0.19.0) history.
I've been active in the PF forums as well over the past two days, and according to the comments in POP3.pm, mails retrieved with the -pop3_toptoo 1 switch never show in history.
Have you noticed similar behaviour? I don't think this is caused by the interaction of MWP with PF, but I'd really like someone to be able to confirm this history problem... I think I'm going crazy 
|
|
| Back to top |
|
|
Eggman5X
Captain
Joined: Mar 13, 2003
Posts: 699
Location: HOU TX USA
|
| Posted: Tue Jun 17, 2003 11:09 am Post subject: But what's the point? |
|
|
Time for the old Eggman to learn a new trick, is it?
Gary's writeup looks brilliant as usual, and I've read all the docs at popfile.sourceforge.net, but I'm not sure I get the point of all this. What's "a Bayesian filter" and why do I want one? Is just for grouping / organization or have I missed something?
What I mean is, my mail client already lets me create mailboxes and folders out my wazoo, route incoming messages to specific folders (or perform up to 17 other actions, including passing the message to a user-defined script, which could add a 'classification' header, or open an input dialog to prompt for one) based on multiple criteria including headers, (individually or as one "head",) body content, date, size, etc.
Also note that at this point MW catches 100% of SPAM, with no falsies. That's for me, of course. I realize your mileage may vary.
It seems to me that this is just another systray / background process to drain more system resources ... or does it have to be "restarted" each time mail is checked? The mail client (of your choice) still needs to "filter" according to the "classification" header, so you've got to be using a mail client that has at least "basic" filtering capabilities to begin with, am I right?
What about B9? Where does it go in the mail handling chain? Is it:
| Code: |
POPfile ->> Mailwasher ->> B9 ->> client
|
... or something else? Does B9 know - or care - about this "classification" header? Will it see it as "non-standard" and a potential threat to be altered or removed?
At this point I can't tell if I'm the dumbest thing standing upright on two legs, or if I have accidentally stumbled onto the ultimate, close-as-can-be-to-perfect email poweruser combo (for less than $50 over three years, btw).
Oh ... just one more thing ...
| Quote: |
(Hey, this is Perl, and we all know that TMTOWTDI, right?  )
|
Er - uh - well, no, even though I spend most of my day in Perl, I guess we don't all know that. "The Monkey Takes Olives With Tequila Drinks Instead" perhaps? 
Best to all, and TIA for the feedback.
_________________
Lightly scrambled, over-easy and stuffed with all sorts of goodies.
|
|
| Back to top |
|
|
Ikeb
Special Response Team
Forums Admin
Joined: Apr 20, 2003
Posts: 16535
|
|
| Back to top |
|
|
IP: 217.136.*.*
Guest
|
| Posted: Tue Jun 17, 2003 11:38 am Post subject: Re: But what's the point? |
|
|
| Eggman5X wrote: |
(snip)
It seems to me that this is just another systray / background process to drain more system resources ... or does it have to be "restarted" each time mail is checked? The mail client (of your choice) still needs to "filter" according to the "classification" header, so you've got to be using a mail client that has at least "basic" filtering capabilities to begin with, am I right?
|
I keep it running in the background all the time, just as MWP sits in the background too. Yes, it probably drains resources, but doesn't spam do too?
About the filtering : I use PF to add extra clues for MWP to filter on. I created an extra filter in MWP that looks for the spam classification header that you can have PF add to mails. While MWP itself already does a wonderful job, combined with PF I get very good results. If a mail is accidentally tagged by PF to be spam, it's likely to be cleared again by my MWP friends list while I retrain PF. After the mails are processed by MWP (deleted in my case) they get send to my final mail client.
| Eggman5X wrote: |
What about B9? Where does it go in the mail handling chain? Is it:
| Code: |
POPfile ->> Mailwasher ->> B9 ->> client
|
... or something else? Does B9 know - or care - about this "classification" header? Will it see it as "non-standard" and a potential threat to be altered or removed?
|
I don't use B9 so if you strip B9 from that setup you state there, you have exactly what I have running now.
|
|
| Back to top |
|
|
Ikeb
Special Response Team
Forums Admin
Joined: Apr 20, 2003
Posts: 16535
|
| Posted: Tue Jun 17, 2003 11:44 am Post subject: Re: But what's the point? |
|
|
| Eggman5X wrote: | Time for the old Eggman to learn a new trick, is it?
Gary's writeup looks brilliant as usual, and I've read all the docs at popfile.sourceforge.net, but I'm not sure I get the point of all this. What's "a Bayesian filter" and why do I want one? Is just for grouping / organization or have I missed something?
|
Well it certainly looks like your sense of humour is brilliant as usual. Check out A Plan for SPAM and let us know what you think.
| Quote: | What about B9? Where does it go in the mail handling chain? Is it:
| Code: |
POPfile ->> Mailwasher ->> B9 ->> client
|
|
I agree that this "overview" level of documentation isn't there ... yet.
| Quote: | | ... or something else? Does B9 know - or care - about this "classification" header? Will it see it as "non-standard" and a potential threat to be altered or removed? |
I wouldn't think so. B9 parses a number of things in HTML formatted documents but to my knowledge doesn't parse the header.
| Quote: | | At this point I can't tell if I'm the dumbest thing standing upright on two legs, or if I have accidentally stumbled onto the ultimate, close-as-can-be-to-perfect email poweruser combo (for less than $50 over three years, btw). |
I hope it's the latter. 
| Quote: | Oh ... just one more thing ...
| Quote: |
(Hey, this is Perl, and we all know that TMTOWTDI, right? )
|
Er - uh - well, no, even though I spend most of my day in Perl, I guess we don't all know that. "The Monkey Takes Olives With Tequila Drinks Instead" perhaps? |
Yeah, some of these acronyms are way out there eh?  Even PERL aficionados have more to learn it seems.  Acronym Finder had no problem finding the more plausible "There's More Than One Way To Do It".
|
|
| Back to top |
|
|
TalonTSi
Corporal
Joined: Mar 16, 2003
Posts: 55
Location: Canada
|
|
| Back to top |
|
|
gary
Lieutenant
Premium Member
Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
|
| Posted: Tue Jun 17, 2003 1:21 pm Post subject: |
|
|
Wow, what a string! Let's see:
| Quote: | As you seem to be quite up to date on both MWP and PF, I noticed that not all mails that show up in MWP show up in the PF (0.19.0) history.
I've been active in the PF forums as well over the past two days, and according to the comments in POP3.pm, mails retrieved with the -pop3_toptoo 1 switch never show in history.
Have you noticed similar behaviour? I don't think this is caused by the interaction of MWP with PF, but I'd really like someone to be able to confirm this history problem... I think I'm going crazy |
Yes, you're exactly right. If you rely solely on the pop3_toptoo option to handle the messages without the hack, then any message retrieved with a "TOP x y" command (which is all of them, from MailWasher) will not be included in the history. The short hack I gave you will bypass this behavior for "TOP x 9999" commands, which MailWasher often issues. If we were to bypass this behavior for all TOP commands, but you have your "spam throttle" set low and you're not using CFS (which forces 200 lines), then accuracy would suffer. Would you like me to change the hack to handle shorter messages, say 100 or 200 lines? I used to use POPFile on pretty short messages, and it still had a fairly good success rate. Let me know what you think.
| Quote: | | What's "a Bayesian filter" and why do I want one? Is just for grouping / organization or have I missed something? |
It's a statistical filter, that categorizes e-mail by the probability of having certain words in a particular type of mail. In the case of POPFile, it also analyzes typical spam techniques, such as invisible writing, excessive commenting, etc. The advantage of this system is that, once you train it, it's very accurate, and handles "fuzzy" message sorting. So, you could have all of your e-mail regarding product XYZ routed to a particular folder for you automatically. The reason this is great is because the e-mail might not always come from the same person, go to the same group, have the same title, or even the same keywords that you have set your mail rules up to handle on your client, but the Bayesian filter will still manage to categorize it. You tell it about the ones that it misses, and it gets smarter!
Bayesian filtering is quite accurate, and, importantly, each person customizes it for the type of spam that they receive. My accuracy is typically better than 98%. If used correctly, it will add little to false positives, but do a better job of detecting spam than the RegExp filter & heuristic combination alone, at least in my experience.
| Quote: | What about B9? Where does it go in the mail handling chain? Is it:
POPfile ->> Mailwasher ->> B9 ->> client |
Hmm, I guess I would show the chain as:
-> WebMail Gateway -> POPFile -> B9 -> MailWasher/Mail Client
with any one of those parts being optional. By the way, keep in mind that B9 alters the body of the message, so if you are Beta testing CFS, it will goof up the results!
Re: TMDOWTDI - sorry, I didn't think I would catch you off guard with that one! It's the very first thing you see on the top of the front cover of O'Reilly's Perl book, and an expression often used by Larry Wall, so it's sprinkled around the forums and Web sites.
Oops! More later - I'm off to work!
_________________
Gary
|
|
| Back to top |
|
|
TalonTSi
Corporal
Joined: Mar 16, 2003
Posts: 55
Location: Canada
|
| Posted: Tue Jun 17, 2003 1:34 pm Post subject: |
|
|
Eggman5X, this is from a different thread but I thought it to be relevant here...
| Eggman5X wrote: | | Another "YES! YES! YES!" for auto-delete log. Even though I don't auto-delete now, this might be the safety net that tempts me to try it. |
An added benefit of daisy-chaining POPfile with MWP is that it will keep an auto-delete history for you! All mail passing through POPfile is logged in POPfile history... um, with the exception of the ones that, well, aren't, as we've mentioned above! 
_________________
--Darren.
|
|
| Back to top |
|
|
IP: 80.200.*.*
Guest
|
| Posted: Tue Jun 17, 2003 2:39 pm Post subject: |
|
|
| gary wrote: |
Yes, you're exactly right. If you rely solely on the pop3_toptoo option to handle the messages without the hack, then any message retrieved with a "TOP x y" command (which is all of them, from MailWasher) will not be included in the history. The short hack I gave you will bypass this behavior for "TOP x 9999" commands, which MailWasher often issues. If we were to bypass this behavior for all TOP commands, but you have your "spam throttle" set low and you're not using CFS (which forces 200 lines), then accuracy would suffer. Would you like me to change the hack to handle shorter messages, say 100 or 200 lines? I used to use POPFile on pretty short messages, and it still had a fairly good success rate. Let me know what you think.
|
The weird thing however, is that I'm using the hacked POP3.pm file, and all check mail commands are coming from MWP. So, unless I miss a very crucial point here, if this hack works, it should either have the effect of adding all mails to the PF history, or none at all.
What strikes me as strange is the "which MailWasher often issues" line in your post. Would there be any reason for MWP not to issue a TOP x 9999 command? Wait.. when you retreive a full mail through a preview maybe?
If we change the hack to include TOP x 99999 to TOP x 9999999 as well, would that solve it?
Note : I'm not a programmer, as you probably can see
|
|
| Back to top |
|
|
Eggman5X
Captain
Joined: Mar 13, 2003
Posts: 699
Location: HOU TX USA
|
| Posted: Tue Jun 17, 2003 2:53 pm Post subject: |
|
|
Ikeb:
Thanks for the reply and two really interesting links. "Acronym Finder" is already bookmarked and I'm crwaling through "A Plan For Spam" in another 'tab'.
P.S. I'm an old time (edible) SPAM aficionado myself ... and not just in sandwiches:
| Code: |
Dice up one can SPAM and fry lightly (3-4 minutes).
Drain excess grease on paper towels, return to pan.
Add one 10-16 oz. can of your favorite brand of chili, let simmer over low heat, stirring now and then.
Prepare one box Mac&Cheese per directions (cheapest available is fine), then add to SPAM/chili mixture.
Mix together and serve warm.
Serves 4-8 (if I don't get to it first!)
|
But buy a treadmill first. You'll need it.
--- --- --- ---
Gary:
You lost me on the chain - or maybe we have different setups - but MW sees my mail, and handles it - long before B9. In fact, MW and B9 don't even know the other exist. Did I do something dumb again?
I would LOVE to be in on the beta testing for CFS, even if I have to turn off B9 for a while. There's little I can mention here that I crave more than a good Chicken Fried Steak (with lots of cream gravy, of course.) 
But seriously, you mean the new Firetrust subscription based thingie, right? Yeah, me too. If another hand is needed, let me know who to contact, or have them contact me. Anything I can do to beat the spammers is time well spent. Heck, I'd even turn off my blacklist and filters for that.
| Quote: |
It's the very first thing you see on the top of the front cover of O'Reilly's Perl book ...
|
The only O'Reilly Perl book I've got is the Camel head (Nutshell) ... you must be talking about another one. Of course, I learned Perl from a "Dummies" book (by Paul E. Hoffman) without knowing any variant of C or Java or any other "modern" langauge (and all those 1960's era languages I knew so well - C----, F------, A-------- - weren't much help), so what do I really know?
--- --- --- ---
TalonTSi:
LOLOMAOTF ... or whatever. All are logged - except the ones that aren't. I LOVE IT!
Which, now that I think of it, explains everything. In fact, I may have the ultimate answer to the whole problem. Now everyone, click your heels three times and repeat today's affirmation after me:
There's no mail that's SPAM. There's just stuff I choose not to read.
--- --- --- ---
All:
Thanks. And pay no attention to the man behind the shower curtain.
_________________
Lightly scrambled, over-easy and stuffed with all sorts of goodies.
|
|
| Back to top |
|
|
gary
Lieutenant
Premium Member
Joined: Dec 22, 2002
Posts: 260
Location: Dallas/Ft. Worth, USA
|
| Posted: Tue Jun 17, 2003 3:46 pm Post subject: |
|
|
| guest wrote: | | The weird thing however, is that I'm using the hacked POP3.pm file, and all check mail commands are coming from MWP. So, unless I miss a very crucial point here, if this hack works, it should either have the effect of adding all mails to the PF history, or none at all. |
Not exactly. There will still be some e-mails that MW uses "TOP x 20" (or TOP x 200, if you are using CFS) to retrieve. I'm not sure why MailWasher sometimes uses 9999 for the number of lines, and sometimes the setting in your "Spam Throttle", but that's what I've noticed. Someone else who's a little more knowledgeable about MW (or a little more observant than me!) could probably tell us. 20 lines is not really enough for POPFile to work with accurately. That's normally enough to grab the header, but very little of anything else. So, POPFile will recognize spam if you get it from the same person over and over, but if you get it from someone new, it probably won't classify it correctly.
| guest wrote: | | If we change the hack to include TOP x 99999 to TOP x 9999999 as well, would that solve it? |
If y'all (<- that's Texan) would like to get both accuracy and the e-mails in your history, we could do this:
1. Crank up your Spam Throttle a little -- say, to 200 lines
2. Change the original hack that I posted to keep a history of e-mails with 200 lines or more
I'm not saying that 100 lines of a message would not work though, as much spam tends to be fairly short anyway. What do y'all (<- there it is again) think?
| Eggman5X wrote: | | P.S. I'm an old time (edible) SPAM aficionado myself ... and not just in sandwiches |
Hehehe -- I feel my arteries hardening already! Love it. I agree with your chicken fried steak with lots of gravy! I have to break down every once in a while and have one.
| Eggman5X wrote: | | You lost me on the chain - or maybe we have different setups - but MW sees my mail, and handles it - long before B9. In fact, MW and B9 don't even know the other exist. Did I do something dumb again? |
No, not at all! You didn't set B9 up as a proxy in MW, that's all! You don't need to use it, but B9 does some really nice things which increases the effectiveness of any filters you have. It will decode escaped (Unicode) characters, Base64, strip out all of the nonsense HTML and tags - in essence, all the crap that spammers use to try to get around filters. It pretty much gives you what you see in the normal message preview in MW, which I wish the filters had access to.
| Eggman5X wrote: | | I would LOVE to be in on the beta testing for CFS [...] But seriously, you mean the new Firetrust subscription based thingie, right? |
Yep, that's it! I don't know if they need any more people at this point, but you might try dropping a note to cfs-beta@firetrust.com.
| Eggman5X wrote: | | The only O'Reilly Perl book I've got is the Camel head (Nutshell) ... you must be talking about another one. |
Yeah, it was just the plain old "Programming Perl", or "Camel" book, that I was referring to.
Wow, I get really long winded with these posts, don't I....
_________________
Gary
|
|
| Back to top |
|
|
|
|
|
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
|
Powered by phpBB © 2001 phpBB Group
|
Forum FAQ
Search
Usergroups
Profile
Login to check your private messages
Login
