From d283785cb27174321935a32d98bdc88c98ac789d Mon Sep 17 00:00:00 2001 From: Nick Bebout Date: Sat, 12 Mar 2011 02:44:23 +0000 Subject: [PATCH] 1.163 --- CREDITS | 6 ++ ChangeLog | 13 ++- FAQ | 64 ++++++++++++--- README | 17 +++- TODO | 10 ++- VERSION | 2 +- aaa | 93 +++++++++++++++++++++ aaa2 | 93 +++++++++++++++++++++ freshmeat_submition | 2 +- imapsync | 192 ++++++++++++++++++++++++++------------------ tests.sh | 162 ++++++++++++++++++++++++++----------- 11 files changed, 508 insertions(+), 146 deletions(-) create mode 100644 aaa create mode 100644 aaa2 diff --git a/CREDITS b/CREDITS index 36d7b33..ee5b552 100644 --- a/CREDITS +++ b/CREDITS @@ -1,5 +1,11 @@ #!/bin/cat +Robert Terzi +Seems to have the mbox vs Maildir/ problem +and gave the --regexmess 's/\AFrom \w .*\n//' +solution. + + Subbarao & Prasanthi Punnamaraju Asked how to move emails from InBox to a sub-folder. diff --git a/ChangeLog b/ChangeLog index fc71551..2a79b9d 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,15 +1,24 @@ RCS file: RCS/imapsync,v Working file: imapsync -head: 1.161 +head: 1.163 branch: locks: strict access list: symbolic names: keyword substitution: kv -total revisions: 161; selected revisions: 161 +total revisions: 163; selected revisions: 163 description: ---------------------------- +revision 1.163 +date: 2006/03/24 04:18:58; author: gilles; state: Exp; lines: +61 -51 +Added ask_for_password() function +FLAGS INTERNALDATE and RFC822.SIZE fetch together +---------------------------- +revision 1.162 +date: 2006/03/22 03:45:05; author: gilles; state: Exp; lines: +59 -31 +Added --authuser1 --authuser2 options +---------------------------- revision 1.161 date: 2006/03/20 00:09:41; author: gilles; state: Exp; lines: +12 -42 Fixed default auth to CRAM-MD5 diff --git a/FAQ b/FAQ index 8334e09..be74b85 100644 --- a/FAQ +++ b/FAQ @@ -27,32 +27,44 @@ b) Use the --syncinternaldates option and keep using Eudora :-) ======================================================================= Q. Does imapsync support IMAP over TLS (IMAPS)? -R. Not natively. -but, 2 ways, at least : +R. Yes natively since release 1.161. +still, 2 ways, at least : + +a) Use --ssl1 and/or --ssl2 options + +--ssl1 tells imapsync to use ssl on host1 +--ssl2 tells imapsync to use ssl on host2 -a) There are patches in patches/ directory to do this - in imapsync code. - I tried to include them but it broke my tests. - It works but not always. b) Use stunnel http://www.stunnel.org/ Assuming there is an imaps (993) server on imap.foo.org, on your localhost machine (or bar machine) run : stunnel -c -d imap -r imap.foo.org:imaps - or + or using names instead of numbers stunnel -c -d 143 -r imap.foo.org:993 then use imapsync on localhost (or bar machine) imap (143) port. +======================================================================= +Q: How to have an imaps server ? +R. +a) Install one + +b) or use stunnel : + Assuming there is an imap (143) server on localhost + stunnel -d 993 -r 143 -f + +c) or use stunnel on inetd + imaps stream tcp nowait cyrus /usr/sbin/stunnel -s cyrus -p /etc/ssl/certs/imapd.pem -r localhost:imap2 ====================================================================== Q. I want the --folder 'MyFolder' option be recurse. R. Do not use the --folder option. Instead, use --include '^MyFolder' - Folder "MyFolder" and all its subfolders will be handled. + Then the folder "MyFolder" and all its subfolders will be handled. ====================================================================== Q. I have moved from Braunschweig to Graz, so I would like to have my whole @@ -106,9 +118,9 @@ messages with the shift key. b) With imapsync: ----------------- -you have to calculate the day of year (and -add 365). For example, running it today, Sat Mar 11 -13:06:01 CET 2006: +You have to calculate the day of year (and +add 365). For example, running it today, +Sat Mar 11 13:06:01 CET 2006: imapsync ... --host1 imap.truc.org --host2 imap.trac.org \ @@ -131,6 +143,36 @@ Also, you must take imapsync 1.159 at least since I tested what I just wrote above and found 2 bugs about --mindate --maxdate options behavior. + +======================================================================= +Q. My imap server does not accept a message and warns + "Invalid header". What is the problem ? + +R. You fall in the classical mbox versus Maildir/ format + problem. May be you use a misconfigured procmail rule. + +A header beginning like the following one is in the mbox +format, header line 1 has no colon behind "From", header +lines 2 through N do have a colon : + +From foo@yoyo.org Sat Jun 22 01:10:21 2002 +Return-Path: +Received: ... + +Any Maidir/ configured imap server may refuse this message +since its header is invalid. The first "From " line is not +valid. It lacks a colon character ":". +To solve this problem you have several solutions a) or b): + +a) Remove these first "From " line manually for each message +before using imapsync. Don't think to add a colon to this +line since you will end with two "From:" lines (just look at +the other lines) + +b) Run imapsync with the following option : +--regexmess 's/\AFrom \w .*\n//' + + ======================================================================= Q. I'm migrating from WU to Cyrus, and the mail folders are under /home/user/mail but the tool copies everything in diff --git a/README b/README index 5d47472..93abe62 100644 --- a/README +++ b/README @@ -2,7 +2,7 @@ NAME imapsync - IMAP synchronization, copy or migration tool. Synchronize mailboxes between two imap servers. Good at IMAP migration. - $Revision: 1.161 $ + $Revision: 1.163 $ INSTALL imapsync works fine under any Unix OS. @@ -116,8 +116,17 @@ SECURITY password in a well protected file (600 or rw-------) is the best solution. - imasync is not protected against sniffers on the network so the - passwords are in plain text. + imasync is not totally protected against sniffers on the network since + passwords may be transfered in plain text in case CRAM-MD5 is not + supported by your imap servers. Use --ssl1 and --ssl2 to enable + encryption on host1 and host2. + + You may authenticate as one user (typically an admin user), but be + authorized as someone else, which means you don't need to know every + user's personal password. Specify --authuser1 "adminuser" to enable this + on host1. In this case, --authmech1 PLAIN will be used, but otherwise, + --authmech1 CRAM-MD5 is the default. Same behavior with the --authuser2 + option. EXIT STATUS imapsync will exit with a 0 status (return code) if everything went @@ -277,5 +286,5 @@ AUTHOR teaching free open and gratis softwares. Don't hesitate to pay him for that services. - $Id: imapsync,v 1.161 2006/03/20 00:09:41 gilles Exp $ + $Id: imapsync,v 1.163 2006/03/24 04:18:58 gilles Exp $ diff --git a/TODO b/TODO index b85486d..6c47c72 100644 --- a/TODO +++ b/TODO @@ -7,10 +7,6 @@ problem is encountered. Post on newsgroup comp.mail.imap when a new release comes. http://groups.google.fr/group/comp.mail.imap -Add features from Herman (patch against 1.139) - - Support for authenticating as different (admin) user - - Support for different auth mechanisms (we needed PLAIN) - Add my amazon wishlist link. Add an --exactsync option to remove target messages @@ -35,6 +31,12 @@ http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html Explain expunge behavior. + +DONE. Add features from Herman (patch against 1.139 and +again with 1.156 by Kjetil Torgrim Homme) + - Support for authenticating as different (admin) user + - Support for different auth mechanisms (we needed PLAIN) + DONE. Make --include --exclude options possibly be a list. DONE. Look at http://freshmeat.net/projects/freshmeat-submit/ diff --git a/VERSION b/VERSION index de6f6fa..3ded366 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.161 +1.163 diff --git a/aaa b/aaa new file mode 100644 index 0000000..545119e --- /dev/null +++ b/aaa @@ -0,0 +1,93 @@ +get options: [1] +$RCSfile: imapsync,v $ $Revision: 1.162 $ $Date: 2006/03/22 03:45:05 $ +Mail::IMAPClient version used here is 2.2.9 +will try to use CRAM-MD5 authentication on host1 +will try to use CRAM-MD5 authentication on host2 +From imap server [loul] port [143] user [tata] +To imap server [plume] port [143] user [tata@est.belle] +Banner : * OK louloutte Cyrus IMAP4 v1.5.19 server ready +loul: no support for AUTHENTICATE CRAM-MD5, using LOGIN +Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA AUTH=PLAIN CRAM-MD5 CRAM-SHA1 IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc. See COPYING for distribution information. +From Buffer I/O : 4096 +To Buffer I/O : 4096 +From capability : QUOTA X-NETSCAPE NAMESPACE X-NON-HIERARCHICAL-RENAME ACL UNSELECT LITERAL+ NO_ATOMIC_RENAME UIDPLUS IMAP4 IMAP4REV1 +To capability : QUOTA STARTTLS NAMESPACE CRAM-SHA1 IDLE AUTH=PLAIN THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN ACL2=UNION CRAM-MD5 IMAP4REV1 THREAD=REFERENCES +From state Authenticated +To state Authenticated +Getting separators +Calling namespace capability +Calling namespace capability +Getting prefix namespace +Calling namespace capability +Getting prefix namespace +Calling namespace capability +From separator and prefix : [.][INBOX.] +To separator and prefix : [.][INBOX.] +From folders : [INBOX.yop.yap] +To folders : [INBOX] [INBOX.2005-INBOX] [INBOX.Draft] [INBOX.Drafts] [INBOX.Sent] [INBOX.Trash] [INBOX.ppp] [INBOX.qqq] [INBOX.yop] [INBOX.yop.yap] [INBOX.yop.yup] [INBOX.yopX] [INBOX.yopX.yap] [INBOX.yopX.yup] [INBOX.zz] +From subscribed folders : [INBOX.yop.yap] +From Folder [INBOX.yop.yap] +removed source prefix : [yop.yap] +inverted separators : [yop.yap] +added target prefix : [INBOX.yop.yap] +To Folder [INBOX.yop.yap] +LIST FROM : 1 messages [1] +LIST TO : 1 messages [7] +++++ From [INBOX.yop.yap] Parse 1 ++++ +Time headers: 0 s +Time sizes : 0 s +Head NUM:10 +FH Date:Fri, 7 Mar 2003 17:00:01 +0100 (CET) +FH From:root@louloutte.dyndns.org (Cron Daemon) +FH Message-Id:<20030307160001.69D18FEEF@louloutte.dyndns.org> +FH Received:by louloutte.dyndns.org (Postfix, from userid 0) id 69D18FEEF; Fri, 7 Mar 2003 17:00:01 +0100 (CET) +FH Return-Path: +FH Subject:Cron /usr/bin/poff libertynico +FH To:root@louloutte.dyndns.org +FH X-Cron-Env: +FH X-Cron-Env: +FH X-Cron-Env: +FH X-Cron-Env: +FH X-Spam-Level:* +FH X-Spam-Status:No, hits=1.6 required=5.0 tests=NO_MX_FOR_FROM,AWL version=2.20 +F msg 1:ri4E1CAmNkh3rP3Mq15HSw:714 +Time headers: 0 s +++++ To [INBOX.yop.yap] Parse 1 ++++ +Time headers: 0 s +Time sizes : 0 s +Head NUM:10 +TH Date:Fri, 7 Mar 2003 17:00:01 +0100 (CET) +TH From:root@louloutte.dyndns.org (Cron Daemon) +TH Message-Id:<20030307160001.69D18FEEF@louloutte.dyndns.org> +TH Received:by louloutte.dyndns.org (Postfix, from userid 0) id 69D18FEEF; Fri, 7 Mar 2003 17:00:01 +0100 (CET) +TH Return-Path: +TH Subject:Cron /usr/bin/poff libertynico +TH To:root@louloutte.dyndns.org +TH X-Cron-Env: +TH X-Cron-Env: +TH X-Cron-Env: +TH X-Cron-Env: +TH X-Spam-Level:* +TH X-Spam-Status:No, hits=1.6 required=5.0 tests=NO_MX_FOR_FROM,AWL version=2.20 +T msg 7:ri4E1CAmNkh3rP3Mq15HSw:714 +Time headers: 0 s +++++ Verifying [INBOX.yop.yap] -> [INBOX.yop.yap] ++++ ++ key ri4E1CAmNkh3rP3Mq15HSw:714 #1 +Message id [ri4E1CAmNkh3rP3Mq15HSw:714] found in t:INBOX.yop.yap +Setting flags +flags from : \Seen +flags to : \Seen +Looking dates +idate from : 22-Aug-2003 19:16:01 +0200 +idate to : 21-May-2005 12:39:11 +0200 +Message ri4E1CAmNkh3rP3Mq15HSw:714 SZ_GOOD f:1:714 t:7:714 +Time : 0 s +++++ Statistics ++++ +Time : 0 sec +Messages transfered : 0 +Messages skipped : 1 +Total bytes transfered : 0 +Total bytes skipped : 714 +Total bytes error : 0 +Detected 0 errors +Please, rate imapsync at http://freshmeat.net/projects/imapsync/ diff --git a/aaa2 b/aaa2 new file mode 100644 index 0000000..1e57c69 --- /dev/null +++ b/aaa2 @@ -0,0 +1,93 @@ +get options: [1] +$RCSfile: imapsync,v $ $Revision: 1.162 $ $Date: 2006/03/22 03:45:05 $ +Mail::IMAPClient version used here is 2.2.9 +will try to use CRAM-MD5 authentication on host1 +will try to use CRAM-MD5 authentication on host2 +From imap server [loul] port [143] user [tata] +To imap server [plume] port [143] user [tata@est.belle] +Banner : * OK louloutte Cyrus IMAP4 v1.5.19 server ready +loul: no support for AUTHENTICATE CRAM-MD5, using LOGIN +Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA AUTH=PLAIN CRAM-MD5 CRAM-SHA1 IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc. See COPYING for distribution information. +From Buffer I/O : 4096 +To Buffer I/O : 4096 +From capability : QUOTA X-NETSCAPE NAMESPACE X-NON-HIERARCHICAL-RENAME ACL UNSELECT LITERAL+ NO_ATOMIC_RENAME UIDPLUS IMAP4 IMAP4REV1 +To capability : QUOTA STARTTLS NAMESPACE CRAM-SHA1 IDLE AUTH=PLAIN THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN ACL2=UNION CRAM-MD5 IMAP4REV1 THREAD=REFERENCES +From state Authenticated +To state Authenticated +Getting separators +Calling namespace capability +Calling namespace capability +Getting prefix namespace +Calling namespace capability +Getting prefix namespace +Calling namespace capability +From separator and prefix : [.][INBOX.] +To separator and prefix : [.][INBOX.] +From folders : [INBOX.yop.yap] +To folders : [INBOX] [INBOX.2005-INBOX] [INBOX.Draft] [INBOX.Drafts] [INBOX.Sent] [INBOX.Trash] [INBOX.ppp] [INBOX.qqq] [INBOX.yop] [INBOX.yop.yap] [INBOX.yop.yup] [INBOX.yopX] [INBOX.yopX.yap] [INBOX.yopX.yup] [INBOX.zz] +From subscribed folders : [INBOX.yop.yap] +From Folder [INBOX.yop.yap] +removed source prefix : [yop.yap] +inverted separators : [yop.yap] +added target prefix : [INBOX.yop.yap] +To Folder [INBOX.yop.yap] +LIST FROM : 1 messages [1] +LIST TO : 1 messages [7] +++++ From [INBOX.yop.yap] Parse 1 ++++ +Time headers: 1 s +Time fir : 0 s +Head NUM:10 +FH Date:Fri, 7 Mar 2003 17:00:01 +0100 (CET) +FH From:root@louloutte.dyndns.org (Cron Daemon) +FH Message-Id:<20030307160001.69D18FEEF@louloutte.dyndns.org> +FH Received:by louloutte.dyndns.org (Postfix, from userid 0) id 69D18FEEF; Fri, 7 Mar 2003 17:00:01 +0100 (CET) +FH Return-Path: +FH Subject:Cron /usr/bin/poff libertynico +FH To:root@louloutte.dyndns.org +FH X-Cron-Env: +FH X-Cron-Env: +FH X-Cron-Env: +FH X-Cron-Env: +FH X-Spam-Level:* +FH X-Spam-Status:No, hits=1.6 required=5.0 tests=NO_MX_FOR_FROM,AWL version=2.20 +F msg 1:ri4E1CAmNkh3rP3Mq15HSw:714 +Time headers: 0 s +++++ To [INBOX.yop.yap] Parse 1 ++++ +Time headers: 0 s +Time fir : 0 s +Head NUM:10 +TH Date:Fri, 7 Mar 2003 17:00:01 +0100 (CET) +TH From:root@louloutte.dyndns.org (Cron Daemon) +TH Message-Id:<20030307160001.69D18FEEF@louloutte.dyndns.org> +TH Received:by louloutte.dyndns.org (Postfix, from userid 0) id 69D18FEEF; Fri, 7 Mar 2003 17:00:01 +0100 (CET) +TH Return-Path: +TH Subject:Cron /usr/bin/poff libertynico +TH To:root@louloutte.dyndns.org +TH X-Cron-Env: +TH X-Cron-Env: +TH X-Cron-Env: +TH X-Cron-Env: +TH X-Spam-Level:* +TH X-Spam-Status:No, hits=1.6 required=5.0 tests=NO_MX_FOR_FROM,AWL version=2.20 +T msg 7:ri4E1CAmNkh3rP3Mq15HSw:714 +Time headers: 0 s +++++ Verifying [INBOX.yop.yap] -> [INBOX.yop.yap] ++++ ++ key ri4E1CAmNkh3rP3Mq15HSw:714 #1 +Message id [ri4E1CAmNkh3rP3Mq15HSw:714] found in t:INBOX.yop.yap +Setting flags +flags from : \Seen +flags to : \Seen +Looking dates +idate from : 22-Aug-2003 19:16:01 +0200 +idate to : 21-May-2005 12:39:11 +0200 +Message ri4E1CAmNkh3rP3Mq15HSw:714 SZ_GOOD f:1:714 t:7:714 +Time : 0 s +++++ Statistics ++++ +Time : 1 sec +Messages transfered : 0 +Messages skipped : 1 +Total bytes transfered : 0 +Total bytes skipped : 714 +Total bytes error : 0 +Detected 0 errors +Please, rate imapsync at http://freshmeat.net/projects/imapsync/ diff --git a/freshmeat_submition b/freshmeat_submition index 6a21cfa..608b24f 100644 --- a/freshmeat_submition +++ b/freshmeat_submition @@ -1,5 +1,5 @@ Project: imapsync -Version: 1.159 +Version: 1.161 Release-Focus: Major bugfixes Hide: Y Home-Page-URL: http://www.linux-france.org/prj/imapsync/ diff --git a/imapsync b/imapsync index 1779f91..b724f2c 100755 --- a/imapsync +++ b/imapsync @@ -6,7 +6,7 @@ imapsync - IMAP synchronization, copy or migration tool. Synchronize mailboxes between two imap servers. Good at IMAP migration. -$Revision: 1.161 $ +$Revision: 1.163 $ =head1 INSTALL @@ -136,8 +136,19 @@ dangerous because of the 'ps auxwwwwe' command. So, saving the password in a well protected file (600 or rw-------) is the best solution. -imasync is not protected against sniffers on the network so -the passwords are in plain text. +imasync is not totally protected against sniffers on the +network since passwords may be transfered in plain text in +case CRAM-MD5 is not supported by your imap servers. Use +--ssl1 and --ssl2 to enable encryption on host1 and host2. + +You may authenticate as one user (typically an admin user), +but be authorized as someone else, which means you don't +need to know every user's personal password. Specify +--authuser1 "adminuser" to enable this on host1. In this +case, --authmech1 PLAIN will be used, but otherwise, +--authmech1 CRAM-MD5 is the default. Same behavior with the +--authuser2 option. + =head1 EXIT STATUS @@ -325,7 +336,7 @@ Gilles LAMIRAL earn his living writing, installing, configuring and teaching free open and gratis softwares. Don't hesitate to pay him for that services. -$Id: imapsync,v 1.161 2006/03/20 00:09:41 gilles Exp $ +$Id: imapsync,v 1.163 2006/03/24 04:18:58 gilles Exp $ =cut @@ -372,14 +383,14 @@ my( $timestart, $timeend, $timediff, $timesize, $timebefore, $ssl1, $ssl2, - $authusing1, $authusing2, + $authuser1, $authuser2, $authmech1, $authmech2, ); use vars qw ($opt_G); # missing code for this will be option. -$rcs = ' $Id: imapsync,v 1.161 2006/03/20 00:09:41 gilles Exp $ '; +$rcs = ' $Id: imapsync,v 1.163 2006/03/24 04:18:58 gilles Exp $ '; $rcs =~ m/,v (\d+\.\d+)/; $VERSION = ($1) ? $1 : "UNKNOWN"; @@ -416,8 +427,8 @@ $error=0; my $banner = join("", '$RCSfile: imapsync,v $ ', - '$Revision: 1.161 $ ', - '$Date: 2006/03/20 00:09:41 $ ', + '$Revision: 1.163 $ ', + '$Date: 2006/03/24 04:18:58 $ ', "\n", "Mail::IMAPClient version used here is ", $VERSION_IMAPClient,"\n" @@ -459,10 +470,12 @@ if ($justconnect) { my $to = (); $from = connect_imap($host1, $port1); - print "From software : ", ($from->Report())[0]; + + + print "From software : ", server_banner($from); print "From capability : ", join(" ", $from->capability()), "\n"; $to = connect_imap($host2, $port2); - print "To software : ", ($to->Report())[0]; + print "To software : ", server_banner($to); print "To capability : ", join(" ", $to->capability()), "\n"; $from->logout(); $to->logout(); @@ -474,12 +487,15 @@ $user2 || missing_option("--user2"); if(defined($authmd5) and not($authmd5)) { $authmech1 ||= 'PLAIN'; - $authmech2 ||= 'PLAIN'; + $authmech2 ||= 'PLAIN'; }else{ - $authmech1 ||= 'CRAM-MD5'; - $authmech2 ||= 'CRAM-MD5'; + $authmech1 ||= $authuser1 ? 'PLAIN' : 'CRAM-MD5'; + $authmech2 ||= $authuser2 ? 'PLAIN' : 'CRAM-MD5'; } +$authuser1 ||= $user1; +$authuser2 ||= $user2; + print "will try to use $authmech1 authentication on host1\n"; print "will try to use $authmech2 authentication on host2\n"; @@ -495,21 +511,29 @@ $fastio2 = (defined($fastio2)) ? $fastio2 : 1; print "From imap server [$host1] port [$port1] user [$user1]\n"; print "To imap server [$host2] port [$port2] user [$user2]\n"; + +sub ask_for_password { + my ($user, $host) = @_; + print "What's the password for $user\@$host? "; + ReadMode 2; + my $password = <>; + chomp $password; + printf "\n"; + ReadMode 0; + return $password; +} + + $password1 || $passfile1 || do { - print "What's the password for $user1\@$host1? "; - ReadMode 2; - $password1 = <>; chop $password1; - printf "\n"; ReadMode 0; + ask_for_password($authuser1 || $user1, $host1); }; $password1 = (defined($passfile1)) ? firstline ($passfile1) : $password1; $password2 || $passfile2 || do { - print "What's the password for $user2\@$host2? "; - ReadMode 2; - $password2 = <>; chop $password2; - printf "\n"; ReadMode 0; + ask_for_password($authuser2 || $user2, $host2); }; + $password2 = (defined($passfile2)) ? firstline ($passfile2) : $password2; my $from = (); @@ -518,29 +542,26 @@ my $to = (); $timestart = time(); $timebefore = $timestart; -$fastio1 = 1; -$fastio2 = 1; - $debugimap and print "From connection\n"; $from = login_imap($host1, $port1, $user1, $password1, - $debugimap, $timeout, $fastio1, $ssl1, $authmech1); + $debugimap, $timeout, $fastio1, $ssl1, + $authmech1, $authuser1); $debugimap and print "To connection\n"; $to = login_imap($host2, $port2, $user2, $password2, - $debugimap, $timeout, $fastio2, $ssl2, $authmech2); + $debugimap, $timeout, $fastio2, $ssl2, + $authmech2, $authuser2); -# No history -$from->Clear(2); -$to->Clear(2); +# history $debug and print "From Buffer I/O : ", $from->Buffer(), "\n"; $debug and print "To Buffer I/O : ", $to->Buffer(), "\n"; - sub login_imap { my($host, $port, $user, $password, - $debugimap, $timeout, $fastio, $ssl, $authmech) = @_; + $debugimap, $timeout, $fastio, + $ssl, $authmech, $authuser) = @_; my ($imap); if ($ssl) { my $socssl = new IO::Socket::SSL("$host:$port"); @@ -554,6 +575,7 @@ sub login_imap { } else { $imap = Mail::IMAPClient->new(); } + $imap->Clear(20); $imap->Server($host); $imap->Port($port); $imap->Fast_io($fastio); @@ -561,27 +583,31 @@ sub login_imap { $imap->Uid(1); $imap->Peek(1); $imap->Debug($debugimap); + $timeout and $imap->Timeout($timeout); + if ($ssl) { $imap->State(Mail::IMAPClient::Connected); } else { $imap->connect() or die "Can not open imap connection on [$host] with user [$user] : $@\n"; } - $timeout and $imap->Timeout($timeout); - + print "Banner : ", server_banner($imap); + if ($authmech eq "LOGIN") { # Default mode for Mail::IMAPClient, so don't do anything. } elsif ($imap->has_capability("AUTH=$authmech") - or $imap->has_capability($authmech)) { + or $imap->has_capability($authmech) + ) { $imap->Authmechanism($authmech); $imap->Authcallback(\&plainauth) if $authmech eq "PLAIN"; } else { printf("%s: no support for AUTHENTICATE %s, using LOGIN\n", $imap->Server, $authmech); } + $imap->User($user); + $imap->Authuser($authuser); $imap->Password($password); - #md5auth($imap); $imap->login() or die "Error login : [$host] with user [$user] : $@"; return($imap); } @@ -591,20 +617,29 @@ sub plainauth() { my $imap = shift; my $string = sprintf("%s\x00%s\x00%s", $imap->User, - $imap->User, $imap->Password); + $imap->Authuser, $imap->Password); return encode_base64("$string"); } +sub server_banner { + my $imap = shift; + for my $line ($imap->Results()) { + #print "LR: $line"; + return $line if $line =~ /^\* (OK|NO|BAD)/; + } + return "No banner\n"; + } + -print "From software : ", ($from->Report())[0]; -print "To software : ", ($to->Report())[0]; print "From capability : ", join(" ", $from->capability()), "\n"; print "To capability : ", join(" ", $to->capability()), "\n"; die unless $from->IsAuthenticated(); +print "From state Authenticated\n"; die unless $to->IsAuthenticated(); +print "To state Authenticated\n"; my (@f_folders, @t_folders, %fs_folders); @@ -612,8 +647,6 @@ my (@f_folders, @t_folders, %fs_folders); map { $fs_folders{$_}=1 } $from->subscribed(); - - if (scalar(@folder)) { # folders given by option --folder @f_folders = @folder; @@ -911,16 +944,13 @@ FOLDER: foreach my $f_fold (@f_folders) { my $f_heads = $from->parse_headers($from->Range([@f_msgs]),@useheader) if (@f_msgs) ; $debug and print "Time headers: ", timenext(), " s\n"; - my $f_size = $from->fetch_hash("RFC822.SIZE") if (@f_msgs); - $debug and print "Time sizes : ", timenext(), " s\n"; - #my $f_flags = $from->flags(@f_msgs) ; - #print "Time flags : ", timenext(), " s\n"; - use Data::Dumper; - #print Data::Dumper->Dump([$f_heads]); - #print Data::Dumper->Dump([$f_flags]); + my $f_fir = $from->fetch_hash("FLAGS", + "INTERNALDATE", + "RFC822.SIZE") if (@f_msgs); + $debug and print "Time fir : ", timenext(), " s\n"; foreach my $m (@f_msgs) { - parse_header_msg1($from, $m, $f_heads, $f_size, "F", \%f_hash); + parse_header_msg1($from, $m, $f_heads, $f_fir, "F", \%f_hash); } $debug and print "Time headers: ", timenext(), " s\n"; @@ -928,17 +958,15 @@ FOLDER: foreach my $f_fold (@f_folders) { my $t_heads = $to->parse_headers($to->Range([@t_msgs]),@useheader) if (@t_msgs); $debug and print "Time headers: ", timenext(), " s\n"; - my $t_size = $to->fetch_hash("RFC822.SIZE") if (@t_msgs); - $debug and print "Time sizes : ", timenext(), " s\n"; - #my $t_flags = $to->flags(@t_msgs) ; - #print "Time flags : ", timenext(), " s\n"; - + my $t_fir = $to->fetch_hash("FLAGS", + "INTERNALDATE", + "RFC822.SIZE") if (@t_msgs); + $debug and print "Time fir : ", timenext(), " s\n"; foreach my $m (@t_msgs) { - parse_header_msg1($to, $m, $t_heads, $t_size, "T", \%t_hash); + parse_header_msg1($to, $m, $t_heads, $t_fir, "T", \%t_hash); } $debug and print "Time headers: ", timenext(), " s\n"; - #exit; - + print "++++ Verifying [$f_fold] -> [$t_fold] ++++\n"; # messages in "from" that are not good in "to" @@ -950,7 +978,8 @@ FOLDER: foreach my $f_fold (@f_folders) { MESS: foreach my $m_id (@f_hash_keys_sorted_by_uid) { my $f_size = $f_hash{$m_id}{'s'}; my $f_msg = $f_hash{$m_id}{'m'}; - # print "."; + my $f_idate = $f_hash{$m_id}{'D'}; + if (defined $maxsize and $f_size > $maxsize) { print "+ Skipping msg #$f_msg:$f_size in folder $f_fold (exceeds maxsize limit $maxsize bytes)\n"; $mess_size_total_skipped += $f_msg; @@ -971,24 +1000,12 @@ FOLDER: foreach my $f_fold (@f_folders) { "F message content ended on previous line\n"; my $d = ""; if ($syncinternaldates) { - $d = $from->internaldate($f_msg); + $d = $f_idate; $d = "\"$d\""; $debug and print "internal date from 1: [$d]\n"; } - my $flags_f_rv = $from->flags($f_msg); - my @flags_f; - my $flags_f; - - if (ref($flags_f_rv)) { - @flags_f = @{$flags_f_rv}; - $flags_f = join(" ", @flags_f); - }else{ - $flags_f = ""; - } - - #$flags_f = join(" ", @{$from->flags($f_msg)}); - + my $flags_f = $f_hash{$m_id}{'F'} || ""; # RFC 2060 : This flag can not be altered by the client $flags_f =~ s@\\Recent@@g; @@ -1023,6 +1040,7 @@ FOLDER: foreach my $f_fold (@f_folders) { my $t_size = $t_hash{$m_id}{'s'}; my $t_msg = $t_hash{$m_id}{'m'}; + $debug and print "Setting flags\n"; my (@flags_f,@flags_t); my $flags_f_rv = $from->flags($f_msg); @@ -1043,11 +1061,14 @@ FOLDER: foreach my $f_fold (@f_folders) { $debug and do { print "Looking dates\n"; - my $d_f = $from->internaldate($f_msg); - my $d_t = $to->internaldate($t_msg); + #my $d_f = $from->internaldate($f_msg); + #my $d_t = $to->internaldate($t_msg); + my $d_f = $f_hash{$m_id}{'D'}; + my $d_t = $t_hash{$m_id}{'D'}; print "idate from : $d_f\n", "idate to : $d_t\n"; + #unless ($d_f eq $d_t) { # print "!!! Dates differ !!!\n"; #} @@ -1198,6 +1219,8 @@ sub get_options "ssl2!" => \$ssl2, "authmech1=s" => \$authmech1, "authmech2=s" => \$authmech2, + "authuser1=s" => \$authuser1, + "authuser2=s" => \$authuser2, ); $debug and print "get options: [$opt_ret]\n"; @@ -1216,7 +1239,7 @@ sub get_options sub parse_header_msg1 { - my ($imap, $m_uid, $s_heads, $s_size, $s, $s_hash) = @_; + my ($imap, $m_uid, $s_heads, $s_fir, $s, $s_hash) = @_; my $head = $s_heads->{$m_uid}; my $headnum = scalar(keys(%$head)); @@ -1244,8 +1267,9 @@ sub parse_header_msg1 { # no header so taking everything $headstr = $imap->message_string($m_uid); } - my $size = $s_size->{$m_uid}->{"RFC822.SIZE"}; - #return unless ($size); + my $size = $s_fir->{$m_uid}->{"RFC822.SIZE"}; + my $flags = $s_fir->{$m_uid}->{"FLAGS"}; + my $idate = $s_fir->{$m_uid}->{"INTERNALDATE"}; $size = length($headstr) unless ($size); my $m_md5 = md5_base64($headstr); $debug and print "$s msg $m_uid:$m_md5:$size\n"; @@ -1257,6 +1281,8 @@ sub parse_header_msg1 { } $s_hash->{"$key"}{'5'} = $m_md5; $s_hash->{"$key"}{'s'} = $size; + $s_hash->{"$key"}{'D'} = $idate; + $s_hash->{"$key"}{'F'} = $flags; $s_hash->{"$key"}{'m'} = $m_uid; } @@ -1284,15 +1310,16 @@ Several options are mandatory. --host1 : "from" imap server. Mandatory. --port1 : port to connect on host1. Default is 143. --user1 : user to login on host1. Mandatory. +--authuser1 : user to auth with on host1 (admin user). --password1 : password for the user1. Dangerous, use --passfile1 --passfile1 : password file for the user1. Contains the password. --host2 : "destination" imap server. Mandatory. --port2 : port to connect on host2. Default is 143. --user2 : user to login on host2. Mandatory. +--authuser2 : user to auth with on host2 (admin user). --password2 : password for the user2. Dangerous, use --passfile2 --passfile2 : password file for the user2. Contains the password. --noauthmd5 : don't use MD5 authentification. ---authmd5 : use MD5 authentification. --authmech1 : auth mechanism to use with host1: PLAIN, LOGIN, CRAM-MD5 etc. --authmech2 : auth mechanism to use with host2. See --authmech1 @@ -1401,3 +1428,14 @@ $rcs See http://www.gnu.org/copyleft/gpl.html EOF } + + +package Mail::IMAPClient; + + +sub Authuser { + my $self = shift; + + if (@_) { $self->{AUTHUSER} = shift } + return $self->{AUTHUSER}; +} diff --git a/tests.sh b/tests.sh index 0f62628..0b1f10f 100644 --- a/tests.sh +++ b/tests.sh @@ -1,6 +1,6 @@ #!/bin/sh -# $Id: tests.sh,v 1.47 2006/02/28 03:53:34 gilles Exp gilles $ +# $Id: tests.sh,v 1.50 2006/03/25 22:22:53 gilles Exp $ #### Shell pragmas @@ -71,13 +71,11 @@ loulplume() { if test X`hostname` = X"plume"; then echo3 Here is plume sendtestmessage - #sleep 10 ./imapsync \ --host1 loul --user1 tata \ --passfile1 /var/tmp/secret.tata \ --host2 plume --user2 tata@est.belle \ - --passfile2 /var/tmp/secret.tata \ - --nosyncacls + --passfile2 /var/tmp/secret.tata else : fi @@ -87,7 +85,6 @@ loulloul() { if test X`hostname` = X"plume"; then echo3 Here is plume sendtestmessage - #sleep 10 ./imapsync \ --host1 loul --user1 tata \ --passfile1 /var/tmp/secret.tata \ @@ -107,8 +104,7 @@ plumeloul() { --host1 plume --user1 tata@est.belle \ --passfile1 /var/tmp/secret.tata \ --host2 loul --user2 tata \ - --passfile2 /var/tmp/secret.tata \ - --nosyncacls + --passfile2 /var/tmp/secret.tata else : fi @@ -122,8 +118,7 @@ lp_folder() { --passfile2 /var/tmp/secret.tata \ --folder INBOX.yop --folder INBOX.Trash \ --host1 loul --user1 tata \ - --passfile1 /var/tmp/secret.tata \ - --nosyncacls + --passfile1 /var/tmp/secret.tata else : fi @@ -154,8 +149,7 @@ lp_justfolders() { --folder INBOX.yop --folder INBOX.Trash \ --host1 loul --user1 tata \ --passfile1 /var/tmp/secret.tata \ - --justfolders \ - --nosyncacls + --justfolders else : fi @@ -170,8 +164,7 @@ pl_folder_qqq() { --passfile1 /var/tmp/secret.tata \ --folder INBOX.qqq \ --host2 loul --user2 tata \ - --passfile2 /var/tmp/secret.tata \ - --nosyncacls + --passfile2 /var/tmp/secret.tata else : fi @@ -186,7 +179,6 @@ pl_prefix12() { --folder INBOX.qqq \ --host2 loul --user2 tata \ --passfile2 /var/tmp/secret.tata \ - --nosyncacls \ --prefix1 INBOX.\ --prefix2 INBOX. \ else @@ -223,8 +215,7 @@ pl_folder() { --passfile1 /var/tmp/secret.tata \ --folder INBOX.yop \ --host2 loul --user2 tata \ - --passfile2 /var/tmp/secret.tata \ - --nosyncacls + --passfile2 /var/tmp/secret.tata else : fi @@ -295,13 +286,12 @@ lp_authmd5() { if test X`hostname` = X"plume"; then echo3 Here is plume - perl -I ~gilles/build/Mail-IMAPClient-2.2.8/blib/lib/ \ ./imapsync \ --host2 plume --user2 tata@est.belle \ --passfile2 /var/tmp/secret.tata \ --host1 loul --user1 tata \ --passfile1 /var/tmp/secret.tata \ - --justconnect + --justfoldersizes else : fi @@ -311,13 +301,12 @@ lp_noauthmd5() { if test X`hostname` = X"plume"; then echo3 Here is plume - perl -I ~gilles/build/Mail-IMAPClient-2.2.8/blib/lib/ \ ./imapsync \ --host2 plume --user2 tata@est.belle \ --passfile2 /var/tmp/secret.tata \ --host1 loul --user1 tata \ --passfile1 /var/tmp/secret.tata \ - --justconnect --noauthmd5 + --justfoldersizes --noauthmd5 else : fi @@ -456,7 +445,7 @@ bad_login() bad_host() { ! ./imapsync \ - --host1 localhost --user1 toto@est.belle \ + --host1 badhost --user1 toto@est.belle \ --passfile1 /var/tmp/secret1 \ --host2 badhost --user2 titi@est.belle \ --passfile2 /var/tmp/secret2 @@ -473,25 +462,7 @@ foldersizes() --passfile2 /var/tmp/secret.tata \ --host1 loul --user1 tata \ --passfile1 /var/tmp/secret.tata \ - --justconnect --foldersizes - else - : - fi - -} - - -foldersizes2() -{ - if test X`hostname` = X"plume"; then - echo3 Here is plume - perl -I ~gilles/build/Mail-IMAPClient-2.2.8/blib/lib/ \ - ./imapsync \ - --host2 plume --user2 tata@est.belle \ - --passfile2 /var/tmp/secret.tata \ - --host1 loul --user1 tata \ - --passfile1 /var/tmp/secret.tata \ - --justconnect --foldersizes + --justfoldersizes else : fi @@ -524,8 +495,8 @@ big_transfert_sizes_only() --passfile1 /var/tmp/secret \ --host2 plume --user2 tete@est.belle \ --passfile2 /var/tmp/secret.tete \ - --subscribed --foldersizes --noauthmd5 \ - --justconnect --fast || \ + --subscribed --noauthmd5 \ + --justfoldersizes || \ true } date2=`date` @@ -573,7 +544,6 @@ essnet_mail2_mail() --user2 gilles@softwareuno.com \ --passfile2 /var/tmp/secret.prw \ --noauthmd5 --sep1 / --foldersizes \ - --nosyncacls \ --prefix2 "INBOX/" --regextrans2 's¤INBOX/INBOX¤INBOX¤' } @@ -590,7 +560,7 @@ for user1 in test1 test2 test3; do --passfile2 /var/tmp/secret.prw \ --noauthmd5 --sep1 / --foldersizes \ --prefix2 "INBOX/" --regextrans2 's¤INBOX/INBOX¤INBOX¤' \ - --nosyncacls --debug \ + --debug \ || true done } @@ -604,7 +574,6 @@ essnet_plume2() --passfile1 /var/tmp/secret.prw \ --host2 plume --user2 tata@est.belle \ --passfile2 /var/tmp/secret.tata \ - --nosyncacls \ --noauthmd5 --sep1 / --foldersizes \ --prefix2 INBOX. --regextrans2 's¤INBOX.INBOX¤INBOX¤' } @@ -697,6 +666,102 @@ regexmess() } +flags() +{ + if test X`hostname` = X"plume"; then + echo3 Here is plume + + ./imapsync \ + --host2 plume --user2 tata@est.belle \ + --passfile2 /var/tmp/secret.tata \ + --host1 loul --user1 tata \ + --passfile1 /var/tmp/secret.tata \ + --folder INBOX.yop.yap \ + --dry --debug + + echo 'rm /home/vmail/tata/.yop.yap/cur/*' + else + : + fi +} + + +lp_ssl() { + if test X`hostname` = X"plume"; then + echo3 Here is plume + ./imapsync \ + --host2 plume --user2 tata@est.belle \ + --passfile2 /var/tmp/secret.tata \ + --host1 loul --user1 tata \ + --passfile1 /var/tmp/secret.tata \ + --ssl1 --ssl2 + else + : + fi +} + +lp_authmech_PLAIN() { + if test X`hostname` = X"plume"; then + echo3 Here is plume + ./imapsync \ + --host2 plume --user2 tata@est.belle \ + --passfile2 /var/tmp/secret.tata \ + --host1 loul --user1 tata \ + --passfile1 /var/tmp/secret.tata \ + --justfoldersizes --nofoldersizes \ + --authmech1 PLAIN --authmech2 PLAIN + else + : + fi +} + +lp_authuser() { + if test X`hostname` = X"plume"; then + echo3 Here is plume + ./imapsync \ + --host2 plume --user2 tata@est.belle \ + --passfile2 /var/tmp/secret.tata \ + --host1 loul --user1 tata \ + --passfile1 /var/tmp/secret.tata \ + --justfoldersizes --nofoldersizes \ + --authuser2 tata@est.belle + else + : + fi +} + +lp_authmech_LOGIN() { + if test X`hostname` = X"plume"; then + echo3 Here is plume + ./imapsync \ + --host2 plume --user2 tata@est.belle \ + --passfile2 /var/tmp/secret.tata \ + --host1 loul --user1 tata \ + --passfile1 /var/tmp/secret.tata \ + --justfoldersizes --nofoldersizes \ + --authmech1 LOGIN --authmech2 LOGIN + else + : + fi +} + +lp_authmech_CRAMMD5() { + if test X`hostname` = X"plume"; then + echo3 Here is plume + ./imapsync \ + --host2 plume --user2 tata@est.belle \ + --passfile2 /var/tmp/secret.tata \ + --host1 loul --user1 tata \ + --passfile1 /var/tmp/secret.tata \ + --justfoldersizes --nofoldersizes \ + --authmech1 CRAM-MD5 --authmech2 CRAM-MD5 + else + : + fi +} + + + # mandatory tests run_tests perl_syntax @@ -730,10 +795,15 @@ test $# -eq 0 && run_tests \ lp_skipsize \ lp_skipheader \ lp_regextrans2 \ - foldersizes2 \ foldersizes \ regexmess \ useheader \ + lp_ssl \ + lp_authmech_LOGIN \ + lp_authmech_CRAMMD5 \ + lp_authmech_PLAIN \ + lp_authuser +