ClearFoundation

This how to will configure a public folder for spam so that Zarafa users can "train" the SpamAssassin spam filter to be more accurate. This folder is accessible via the Webmail or Outlook. I assume you already have Zarafa installed and running as per the forum thread - you will also need the Zarafa-Gateway service running

Big thanks to the SME thread here, the steps below are derived from it
wiki.contribs.org/Zarafa_Bayesian_Learning

1. Configure ClearOS LDAP to use Zarafa LDAP schema
   

   Code:

   cp /usr/share/zarafa/zarafa.schema /etc/openldap/schema/ vi /etc/openldap/slapd.conf ## add the following line after the top section include /etc/openldap/schema/zarafa.schema ## restart LDAP service ldap restart




2. Modify webconfig php so that it recognises the Zarafa LDAP parameters
   Edit /var/webconfig/api/User.class.php, add the "zarafa-user" to the core class list. Around line 210:
   

   Code:

   $this->coreclasses = array(         'top',         'posixAccount',         'shadowAccount',         'inetOrgPerson',         'kolabInetOrgPerson',         'hordePerson',         'pcnAccount',         'zarafa-user', );




3. Create the Zarafa Public store (if you haven't already)
   

   Code:

   zarafa-admin -s




4. Create a user account via the webconfig with mail permissions only specifically for collecting the mail. I've used 'SpamAdmin' account below



5. Use LdapAdmin or similar tool to assign a user Zarafa admin account, and add zarafa-user to object class. See screenshots below.
   a. First set your LDAP publish policy to enabled within the webconfig
   b. Use LdapAdmin to login using your LDAP credentials Bind DN, Base, and Bind Password
   c. Right click on your user, and add the Zarafa-user object class to the drop down list on the left hand side
   d. Then change the Zarafa-Admin attribute to 1
   



6. Create ham/spam folders and assign permissions
   Login to Zarafa with an account that has admin rights as assigned above and make two new folders LearnAsSpam and LearnAsHam under: Public folder > Public folders. Set the permissions (right-click folder > Properties > Permission-tab) on both these new folders to:
   
   Spam administration account
   * Folder visible
   * Read items
   * Edit items: all
   * Delete items: all
   
   Everyone (and/or other users/groups you've added at least need:)
   * Folder visible
   * Create items
   * Edit items: none
   * Delete items: none



7. Create a script that will login via IMAP check the folders periodically and pull mail into the ClearOS spam training system. Note that you will need the Zarafa Gateway service running for IMAP access. You can run this on a seperate port if you are still using the ClearOS IMAP. I've assumed that Zarafa Gateway is listening on port 143.
   
   Create /usr/bin/Zarafa-sa-learn.pl
   

   Code:

   #!/usr/bin/perl # # Extract mail from imap server shared folder 'Public folders/LearnAsSpam' & 'Public folders/LearnAsHam' # Orig by dmz@dmzs.com - March 19, 2004 # http://www.dmzs.com/tools/files/spam.phtml # Modified for compatibility with ClearOS spam training Jan 14, 2011, spam training handled by external script # LGPL use Mail::IMAPClient; my $debug=0; my $salearn; my $filterdir="/var/spool/filter/training"; my $spamwebclient="$filterdir/spam-web"; my $notspamwebclient="$filterdir/notspam-web"; # # # # # # # # # # EDIT USER AND PASSWORD (CHECK PORT) # # # # # # # # # # my $imap = Mail::IMAPClient->new( Server=> '127.0.0.1:143',                                   User => 'spamadmin',                                   Password => 'yourpasswordhere',                                   Debug => $debug); if (!defined($imap)) { die "IMAP Login Failed"; } # If debugging, print out the total counts for each mailbox if ($debug) {          my $spamcount = $imap->message_count('Public folders/LearnAsSpam');          print $spamcount, " Spam to process\n";          my $nonspamcount = $imap->message_count('Public folders/LearnAsHam');          print $nonspamcount, " Notspam to process\n" if $debug; } # Process the spam mailbox $imap->select('Public Folders/LearnAsSpam'); my @msgs = $imap->search("ALL"); for (my $i=0;$i <= $#msgs; $i++) {          # export message to file          $imap->message_to_file($spamwebclient.".zarafa.".time.".".$i,$msgs[$i]);          # delete processed message          $imap->delete_message($msgs[$i]);          open("placeholder",'>'.$spamwebclient) or die "Can't create placeholder $!";          close("placeholder"); } $imap->expunge(); $imap->close(); # Process the not-spam mailbox $imap->select('Public Folders/LearnAsHam'); my @msgs = $imap->search("ALL"); for (my $i=0;$i <= $#msgs; $i++) {          $imap->message_to_file($notspamwebclient.".zarafa.".time.".".$i,$msgs[$i]);          # delete processed message          $imap->delete_message($msgs[$i]);          open("placeholder",'>'.$notspamwebclient) or die "Can't create placeholder $!";          close("placeholder"); } $imap->expunge(); $imap->close(); $imap->logout();

   
   Make sure it's executable
   

   Code:

   chmod a+x /usr/bin/Zarafa-sa-learn.pl




8. Create cron script to regularly pull mail from the Public folder into the Spam training system
   Create /etc/cron.d/zarafa-spamassassin
   

   Code:

   # Extract mail from spam folders for training every 10minutes from 8:00 to 22:00 during weekdays */10 8-22 * * 1-5 root /usr/bin/Zarafa-sa-learn.pl




9. Test! you should now be able to login via webmail or Outlook, and drag (move) mail that isnt' being correctly recognised as SPAM into the LearnAsSpam folder and ClearOS will 'auto-learn' it. The same applies for HAM (mail incorreclty marked as SPAM). Note that mail is deleted after it is moved into the Public Folders so keep a copy if you need it.

Enjoy!

Thanks Tim. I'll try this week.

Congratulations

Giving it a try
I'll try to report back with my results.

It seemed like this one chrashed my box... I installed it as per guide, tossed a few hundred mails in the LearnAsSpam, sat back and waited... time went by, nothing happend, then it hit's me... it's past 22, oh well, turned in and the next morning i went of to work. And at 9 I wanted to check up on the folder status, was it empty og was it still full..? Guess what.. I could not get in contact with my server..?? Dang!!! Went home with the thoughts of a long evening of error correcting in mind. My server is an old notebook that serves my needs, and it's alway turned on (Hey it has a server function so it has to be) and as a result it gets a bit hot. Nothing alarming, just about 37 degrees (C). The day before installing from this guide, I had som other work to do on the notebook. I forgot to close the screen down after i was done. Did I mention I have a cat...? He's a one year old helfbreed Maincoon, and he just looooves those hot places.... He lay down on the keyboard, and by doing that also pressed down on the powerbutton... And with a purr from the cat and a hum from the notebook, my server went offline...

Got the cat off the notebook and the notebook powered on again, and presto... the cron job empties the folder as expected, and it seems like the Bayesian learning is taking place.

LOL glad it's working, thanks for the feedback

Tim,

I'm having trouble connecting to LDAP using LdapAdmin. It always returns "LDAP Server Down". I know its up and running. Can you explain a little more on exactly what I need to enter in it.

Thanks

Hi Kevin, is your LDAP publish policy enabled? if not, then you'll only be able to connect via the local loopback interface (127.0.0.1).. You can change this setting from the LDAP menu in the webconfig

Got it... I have some FAT Fingers... Great work.. I'm loving Zarafa.

Thanks for the great work, Tim - on Zarafa especially and this training feature in particular. I'm up and running Zarafa 7.01. Very much appreciate your contribution of so much time and talent.

hi,Tim!

[zarafa 7.01] some wrong when restart ladp before add 'zarafa.schem' to sldap.conf:

[root@mail openldap]# service ldap start

Checking configuration files for slapd: [FAILED]
/etc/openldap/schema/zarafa.schema: line 182 objectclass: AttributeType not found: "mail"
slaptest: bad configuration file!

my sldap.conf:
-----------------------------------------------------------------------------
access to dn.regex="(.*,)?cn=internal,dc=mail,dc=post88,dc=net"
by group/kolabGroupOfNames="cn=admin,cn=internal,dc=mail,dc=post88,dc=net" write
by group/kolabGroupOfNames="cn=maintainer,cn=internal,dc=mail,dc=post88,dc=net" write
by self write
by dn="cn=nobody,cn=internal,dc=mail,dc=post88,dc=net" read #The line 182
by anonymous auth stop
-----------------------------------------------------------------------------
hope your help,thx!