[phpBB Debug] PHP Notice: in file [ROOT]/includes/session.php on line 2208: Array to string conversion
[phpBB Debug] PHP Notice: in file [ROOT]/includes/session.php on line 2208: Array to string conversion
[phpBB Debug] PHP Notice: in file [ROOT]/includes/session.php on line 2208: Array to string conversion
[phpBB Debug] PHP Notice: in file [ROOT]/includes/session.php on line 2208: Array to string conversion
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4688: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3823)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4690: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3823)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4691: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3823)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4692: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3823)
Poco Forums • View topic - A Comprehensive Junk Mail Filtering System

A Comprehensive Junk Mail Filtering System

Discussion on Bayesian and standard junk mail filters

Moderators: Eric, Tomas, robin, Michael

A Comprehensive Junk Mail Filtering System

Postby cssutto » Sat Mar 25, 2006 4:48 pm

Like others who have posted here, I have been very disappointed in the fact that no one has posted a really comprehensive plan for avoiding spam and deleting what does get through.

I have tried several ways in the past. One was a very long list of filters. They were automatic and put addresses and domains in the banned file, many times good names.

So I simplified it and had to do a lot of delete, delete, delete.

But within the past few weeks, I came up with this idea, which so far seems to work.

My first filter is a Pre Download Filter "Harvester Killer on Server" which scans all mail. I assume everyone knows this filter removes from the server all mail that has multiple recipients on the same server. A lot of spam has this trait; that is it is sent to hundreds of addresses on the same server.

My first Incoming Filter is "WireTap", a name I gave the filter that searches for java script, available on the POCO script site.

I am paranoid.

My second Incoming Filter searches for messages addressed to a news group and sends them to the appropriate folder. There are several more of these.

Next is a filter that compares all incoming mail against my address books. If found in the address books, the mail goes directly to the "In" box so no time is wasted on them.

Next is "Junk Message from Sender", a POCO script that matches all mail against the banned senders list. If found, it goes directly to trash. I am doing this for the next 30 days or less just to see what the filter is finding as banned so I do not lose any desirable mail. At the end of the trial period, I will change this to a pre download filter to delete all banned senders. I am alternating this with the filter that looks for "from" in the %junk senders%, which appears to work just as well and should be faster.

I need the test period because I have hundreds and hundreds of banned senders that got there by filters I had in the past and that were not really that well constructed. For instance, I found on a search msn.com banned as a domain. Banning msn.com or aol.com is going to trash a lot of mail that should not be trashed.

For the duration of the test period only, I also double check this filter and the Harvester Killer by looking at all mail with the "view server" window before I download mail. I make note of any names that I want to receive mail from and keep the window open until the download is complete so that if one of my filters zapps the wrong mail, I can manually copy that address and ask the sender to resend.

So far, I have had no need to ask for a resend.

Next is the DBSNL Script. I weighted everything in it at 25.

Next and last at this time is the junk mail filter which is run on all messages that survive to this point.

Mail that gets put in the junk mail box by the junk mail filter is operated on with a button by a script which strips it of all attachments and then sends it to the banned senders list in the junk mail filter. This script is a combination of PJR's "attachment remover" and Bruce Olson's " ban sender and delete".

Since no good mail ever ends up in my junk mailbox, only the bad stuff, it is only a matter of seconds to select all of it with shift+ctrl +click and zap it with the button.

For a short time, I am going to click on each message in the junk mail file one at a time to be sure it is getting all of the attachments.

So far, a typical morning on my first mail check, it will look like this: 75 messages total, 3 spams in the inbox with the good mail, 50 or so trashed by "Harvester Killer on Server", 6 or 8 or so in the trash mail box put there by %junksenders% search and the other 10 or 15 in the junk mail box, again a quick look at the junk mail box and zap them with the button and send them to the banned senders list.

As of last week, I had 26,415 adresses in the banned senders. That may be one reason very few spams get into my inbox. A few will always because the perfect address from a sender not in any of the banned lists or the DBSNL references is going to get by. Up until that point, he is clean.

My only wish so far with this system is that I am not able to instruct the junk mail filter to automatically send junk mail directly to the banned senders list and delete it automatically.

However, the junk mail filter does its job and sends the mail to the junk mail box and until it arrives in that box, it is out of control by any script.

Another interesting thing I picked up while working on spam.

The delete button on POCO or Barca does not get all attachments. It will warn you of the ordinary attachments and ask you whether you want to delete them or not, but there are some that it will not delete.

Here are some examples:
biuf9c2.jpg
pgmplvod.gi
lqpencil.gi
hotel.gi

Therefore, I put PJR's attachment remover in its original form on another button and I run it on every email that I would ordinarily delete with the delete button and then use the delete button to delete it. When I get the time, I will simply add a delete script to PJR's and do away with the standard delete button.

I get a lot of email from business associates who send everything from documents, .jpg and complex AutoCad drawings to birthday cards and it is no fun searching through hundreds of weird names and numbers (PTV-36708, etc.) trying to figure out which attachments I should keep and which I should trash.

Maybe this will help someone work through the problem of spam and maybe not. But for what it is worth, here it is.

And I think the main thing is that with DBNSL and the banned senders list, anyone should be able to handle spam with the tools that come with POCO and Barca.

One last word. The DBSNL and the PJR Attachment Remover are two fantastic scripts. None of this would work well without them.

CSSJR
cssutto
Drop-in Visitor
 
Posts: 7
Joined: Tue Mar 07, 2006 4:25 am

Return to Junk Mail Filtering Help and How-To

Who is online

Users browsing this forum: No registered users and 2 guests

cron