diff --git a/ChangeLog b/ChangeLog index af517f3..660c374 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,17 +1,38 @@ RCS file: RCS/imapsync,v Working file: imapsync -head: 1.463 +head: 1.468 branch: locks: strict - gilles: 1.463 + gilles: 1.468 access list: symbolic names: keyword substitution: kv -total revisions: 463; selected revisions: 463 +total revisions: 468; selected revisions: 468 description: ---------------------------- -revision 1.463 locked by: gilles; +revision 1.468 locked by: gilles; +date: 2011/11/18 01:23:37; author: gilles; state: Exp; lines: +33 -105 +Replaced fetch_hash_2() calls by normal fetch_hash() +---------------------------- +revision 1.467 +date: 2011/11/17 15:00:15; author: gilles; state: Exp; lines: +20 -16 +Subscribe to host2 folders only for those not already subscribed. +---------------------------- +revision 1.466 +date: 2011/11/17 14:31:55; author: gilles; state: Exp; lines: +76 -60 +Use examine() on host1 instead of select(). +---------------------------- +revision 1.465 +date: 2011/11/17 14:02:53; author: gilles; state: Exp; lines: +22 -22 +Changed imapsync homepage from http://www.linux-france.org/prj/imapsync/ to http://imapsync.lamiral.info/ +Fix select behavior on host2 to allow Gmail folders that don't exist but that are listed. +---------------------------- +revision 1.464 +date: 2011/11/14 23:59:46; author: gilles; state: Exp; lines: +8 -8 +Replaced // by ? : equivalent to be perl 5.8 compliant. +---------------------------- +revision 1.463 date: 2011/11/12 21:58:52; author: gilles; state: Exp; lines: +9 -7 --subscribe is on by defaut. ---------------------------- diff --git a/FAQ b/FAQ index 385fcdd..422d489 100644 --- a/FAQ +++ b/FAQ @@ -1,5 +1,5 @@ #!/bin/cat -# $Id: FAQ,v 1.95 2011/11/12 23:41:20 gilles Exp gilles $ +# $Id: FAQ,v 1.97 2011/11/17 22:14:15 gilles Exp gilles $ +------------------+ | FAQ for imapsync | @@ -45,23 +45,27 @@ To fix it just replace single quotes ' by double quotes " ======================================================================= Q. How to install imapsync? -R. http://www.linux-france.org/prj/imapsync/INSTALL +R. Read the INSTALL file in the tarball also available at + http://imapsync.lamiral.info/INSTALL ======================================================================= -Q. How to configure imapsync? +Q. How to configure and run imapsync? -R. http://www.linux-france.org/prj/imapsync/README +R. Read the README and FAQ files in the tarball also available at + http://imapsync.lamiral.info/README + http://imapsync.lamiral.info/FAQ ======================================================================= Q. Can you give some configuration examples? -R. http://www.linux-france.org/prj/imapsync/FAQ +R. The FAQ file contains many examples for several scenarios + http://www.linux-france.org/prj/imapsync/FAQ ======================================================================= Q. How can I have commercial support? R. Buy support from imapsync author and expert: Gilles LAMIRAL -http://ks.lamiral.info/imapsync/#buy_support +http://imapsync.lamiral.info/#buy_support ======================================================================= Q. How can I have gratis support? @@ -125,6 +129,16 @@ Q. Where I can find old imapsync releases? R. Search the internet. +======================================================================= +Q. Where I can find free open and gratis imapsync releases? + +R. Search the internet. + +Q. Is is legal? + +R. Yes, the license permits it + http://imapsync.lamiral.info/COPYING + ======================================================================= Q. How can I try imapsync with the new Mail::IMAPClient 3.xx perl library? @@ -133,8 +147,7 @@ R. - Download latest Mail::IMAPClient 3.xx at - untar it anywhere: tar xzvf Mail-IMAPClient-3.xx.tar.gz - - Download latest imapsync at - http://lamiral.info/~gilles/imapsync/imapsync + - Get any imapsync (latest is better). - run imapsync with perl and -I option tailing to use the perl module Mail-IMAPClient-3.xx. Example: @@ -233,17 +246,19 @@ b) use --idatefromheader to set the internal dates on host2 same as the c) In Maildir boxes, after the sync (too late...), use the script learn/adjust_time.pl to change the internal dates from the "Date:" header. +(this a Unix fix using touch command) -d) Use a better email client. +d) Use a better email client or configure it in order it sorts messages +by sent date. ======================================================================= Q. imapsync calculates 479 messages in a folder but only transfers 400 messages. What's happen? -R1. Unless --useuid is used, imapsync considers the header part +R1. Unless --useuid is used, imapsync considers a header part of a message to identify a message on both sides. - Header part is taken as a whole with "--useheader ALL" or + The header part is whole header with "--useheader ALL" or only specific lines depending on --useheader --skipheader or default values. @@ -256,7 +271,7 @@ The result is that you can have more messages on host1 than on host2. R2. With option --useuid imapsync doesn't use headers to identify messages on both sides but it uses their imap uid. In than case -duplicates are transfered and --delete2 won't work. +duplicates on host1 are transfered on host2. ======================================================================= Q. I need to log every output on a file named log.txt @@ -280,28 +295,36 @@ Q. I run multiple imapsync applications at the same time then get a Is this a potential problem when trying to sync multiple IMAP account in parallel? -R1. No issue with the file imapsync.pid if you don't use its content. -This file can help you to manage multiple runs by sending -signals to the processes (sigterm or sigkill) using their PID, -each run can have its own pid file with --pidfile option. +R1. No issue with the file imapsync.pid if you don't use its content +by yourself. + +This file can help you to manage multiple runs by sending signals +to the processes (sigterm or sigkill) using their PID. +Each run can have its own pid file with --pidfile option. The file imapsync.pid contains the PID of the imapsync process. This file is removed at the end of a normal run. -You can saafely ignore the warning if you don't use imapsync.pid. +You can safely ignore the warning if you don't use imapsync.pid file. ======================================================================= Q. Couldn't create [INBOX.Ops/foo/bar]: NO Invalid mailbox name: INBOX.Ops/foo/bar -Example: -sep1=/ -sep2=. +Let begin by an explanation. -imapsync revert each separator automaticaly. +Example: +sep1 = / +sep2 = . + +imapsync reverts each separator automaticaly. a) All / character coming from host1 are converted to . (convert the separator) b) All . character coming from host1 are converted to / (to avoid intermediate unwanted folder creation). +So +INBOX/Ops.foo.bar (Ops.foo.bar is just one folder name) will be translated to +INBOX.Ops/foo/bar + Sometimes the sep1 character is not valid on host2 (character "/" usualy) R. Try : @@ -316,7 +339,10 @@ any string (including the empty string). ======================================================================= Q. The option --subscribe does not seem to work -R. Use it with --subscribed +R1. Use it with --subscribed + +R2. There is also the --subscribe_all option that subscribe +to all folders on host2. ======================================================================= Q. Does imapsync retain the \Answered and $Forwarded flags? @@ -324,13 +350,14 @@ Q. Does imapsync retain the \Answered and $Forwarded flags? R. It depends on the destination server. a) If the destination server honors the "PERMAENTFLAGS \*" -directive or no PERMAENTFLAGS at all then imapsync synchronises -all flags except the flag \Recent -(RFC 3501 says "This flag can not be altered by the client."). +directive (meaning it accepts any flag) or no PERMAENTFLAGS at all +then imapsync synchronises all flags except the \Recent flag +(RFC 3501 says about \Recent flag "This flag can not be +altered by the client."). b) If the destination server honors the "PERMAENTFLAGS without the -special "\*" (meaning it accepts any flag) then imapsync synchronises -only the flags listed in PERMANENTFLAGS. +special "\*" then imapsync synchronises only the flags listed +in PERMANENTFLAGS. Some imap servers have problems with flags not beginning with the backslash character \ @@ -360,7 +387,10 @@ For example to convert flag $label1 to \label1 Q. I need to keep only a defind list of flags, how can I do? The destination imap server complains about bad flags (Exchange). -R. For example if you want to keep only the following flags +R1. Recent imapsync deals with this issue by filter with PERMANENTFLAGS +automatically. + +R2. For example if you want to keep only the following flags \Seen \Answered \Flagged \Deleted \Draft then use these magic --regexflag options (thanks to Phil): @@ -392,7 +422,7 @@ Any other flag must begin with another character. System flags are just flags defined by an RFC instead of by users. Conclusion, some imap server coders don't read the RFCs (so do I). -recent imapsync deals with this issue by filter with PERMANENTFLAGS +Recent imapsync deals with this issue by filter with PERMANENTFLAGS automatically. ======================================================================= @@ -402,13 +432,7 @@ R. It happens with some servers on the first sync. Also, it was a bug from revision 1.200 to revision 1.207 Solution: run imapsync a second time. imapsync synchronizes flags -on each run unless option --fast is used. - -======================================================================= -Q. imapsync hangs taking up 99.8% cpu right after start, - after printing imapd doesn't support MD5 auth. - -R. Try option --noauthmd5 +on each run. ======================================================================= Q. Some passwords contain * and " characters. Login fails. @@ -443,10 +467,8 @@ What can I do? R. May be spending too much time on the source server, the connection timed out on the destination server. -Try options : - --nofoldersizes --useheader Message-ID --fast +Try options --nofoldersizes -recent imapsync reconnects automatically. ======================================================================= Q. imapsync failed with a "word too long" error from the imap server, @@ -492,28 +514,24 @@ c) or use stunnel on inetd imaps stream tcp nowait cyrus /usr/sbin/stunnel -s cyrus -p /etc/ssl/certs/imapd.pem -r localhost:imap2 ======================================================================= -Q: I'm trying to use imapsync on win32 for gmail, but it requires ssl, -or at least claims to. Imapsync appears to require io-socket-ssl, -which doesn't seem to be available on win32. Are there any other -options? +Q: I'm trying to use imapsync on win32 for gmail and it requires ssl. +Imapsync appears to require IO::Socket::SSL. What can I do? -R: (Q and R come as is from Bryce Walter) +R1: use standalone imapsync.exe, it contains IO::Socket::SSL +Perl module embeded. -I think I'm having success using cygwin perl instead of ActiveState -Perl. I wasn't able to get CPAN working and building IO::Socket::SSL -in ActiveState, but cygwin did all right. I had to force the install -of the Net::SSLeay dependency, because it partially failed one test, -but I think it worked anyway. In order to get working in cygwin, I -installed the entire "perl" category, lynx, ncftp, and lftp (specified -as ftp program in cpan setup). I'm not sure if I needed all those, or -if cpan just kept asking because I didn't have any installed at the -time. Anyway, cpan worked, and I installed all dependencies that -imapsync complained about until it started working. +R2: IO::Socket::SSLio-socket-ssl is available on Win32 +with Strawberry Perl. ======================================================================= Q: Multiple copies when I run imapsync twice ore more. -R. Multiple copies of the emails on the destination server. Some IMAP +R1. You can use option --useuid, imapsync then won't use header lines to +compare messages in folders. Keep in ming it uses a local cache. + + imapsync ... --useuid + +R2. Multiple copies of the emails on the destination server. Some IMAP servers (Domino for example) add some headers for each message transfered. The message is transfered again and again each time you run imapsync. This is bad of course. The explanation is that imapsync @@ -815,49 +833,30 @@ Examples: ======================================================================= Q. I would like to move emails from InBox to a sub-folder called, - say "2005-InBox" based on the date (Like all emails received in the - Year 2005 should be moved to the folder called "2005-InBox"). + say "2010-INBOX" based on the date (Like all emails received in the + Year 2010 should be moved to the folder called "2010-INBOX"). R. 2 ways : a) Manually: ------------ -1) You create a folder INBOX.2005-INBOX +1) You create a folder INBOX.2010-INBOX -2) Mostly every email software allow sorting by date. In inbox, you - select from 1 january to 31 december messages with the shift key. +2) Mostly every email software allow sorting by date. In INBOX, you + select from 1 january to 31 december 2010 messages with the shift key. (in mutt, use ~d) -3) Cut/paste in INBOX.2005-INBOX +3) Cut/paste in INBOX.2010-INBOX b) With imapsync: ----------------- -You have to calculate the day of year (and add 365). For example, -running it today, Sat Mar 11 13:06:01 CET 2006: - -imapsync ... ---host1 imap.truc.org --host2 imap.trac.org \ ---user1 foo --user2 foo \ -... ---maxage 435 --minage 70 \ ---regextrans2 's/^INBOX$/INBOX.2005-INBOX/' \ +imapsync ... \ +--search 'SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010' +--regextrans2 's/^INBOX$/INBOX.2010-INBOX/' \ --folder INBOX - -To know the day of year: - -$ date -Sat Mar 11 13:06:01 CET 2006 - -$ date +%j -070 - -Also, you must take imapsync 1.159 at least since I tested what I just -wrote above and found 2 bugs about --mindate --maxdate options -behavior. - ======================================================================= Q. I want to play with headers line and --regexmess but I want to leave the body as is @@ -997,8 +996,6 @@ imapsync --host1 mail.oldhost.com \ --host2 imap.gmail.com --ssl2 \ --user2 my_email@gmail.com \ --password2 password \ - --useheader 'Message-Id' \ - --prefix2 '[Gmail]/' \ --folder 'INBOX.Sent' \ --regextrans2 's/Sent/Sent Mail/' @@ -1025,6 +1022,8 @@ If your destination imap server doesn't like "[Gmail]" name, just add option: --regextrans2 's/\[Gmail\]/Gmail/' +You can select folders exported to imap within the gmail preferences, +unselect all "System labels" ======================================================================= Q. migrate email from gmail to google apps @@ -1050,6 +1049,7 @@ R. Use --host1 imap.mail.yahoo.com --sep1 '/' --sep1 '/' Can also add --ssl1 to gain encrypted transfer from yahoo. +SSL seems to be mandatory for yahoo (since november 2011) ======================================================================= Q. from Microsoft's Exchange 2007 to Google Apps for your Domain @@ -1178,77 +1178,22 @@ patch saved in ./patches/imapsync-1.337_tobit_V6.patch ======================================================================= -Q. I need to migrate 1250 mailboxes from one cyrus-IMAP server to another -(empty) one. (Box-swap). The passwords are in a MySQL Database. +Q. I need to migrate 1250 mailboxes, passwords are in a MySQL Database. Can you tell me if your script suits my needs? -R. Partially, mailboxes must exist before running imapsync. -May be, Box-swap is not listed in "Failure stories" -nor "Success stories" in the README file. -You have to extract user and password in a csv file. +R. Mailboxes must exist before running imapsync. +You have to extract users logins and passwords in a csv file. See the "HUGE MIGRATION" section in the README file. ====================================================================== Q. From Cyrus to Notes -Juhu! --useheader 'Message-ID' --skipsize does wonders! :) +Default behavior might works. ====================================================================== Q. From cyrus to dbmail -R. (Given by Michael Monnerie, left as is) - -dbmail creates a header like -X-DBMail-PhysMessage-ID:94348 -on messages. There was another problem with cyrus, other headers get -modified also, so I recommend this for a full sync: - -imapsync --host1 cyrushost --user1 u1 --password1 p1 \ - --host2 dbmailhost --user2 u2 --password2 p2 \ - --useheader 'Message-ID' \ - --skipsize \ - --syncinternaldates - -I needed it because there are obviously some -encoding differences between the two servers, and modified headers lead -to all messages being doubled. Examples: - -dbmail: -Content-Type:text/plain; format=flowed; charset=iso-8859-1; -reply-type=original -cyrus: -Content-Type:text/plain; format=flowed; charset="iso-8859-1"; -reply-type=original - -And also the "Received" headers got blanks added in DBmail, etc. - -****************** - -For a full server copy from cyrus to dbmail I used: -imapsync --host1 cyrus --user1 x --authuser1 x --password1 x --ssl1 \ - --host2 dbmail --user2 x --authuser2 x --password2 x \ - --skipheader '(^X-|^Received|^MIME-|^Content-Type|^Disposition-|^From|^Cc|^Reply-|^Subject|^To|^DomainKey).*' \ - --skipsize \ - --sep1 '/' --exclude 'user/demo/Trash' \ - --regextrans2 's/^user.//' --syncinternaldates - -The 'exclude user/demo/Trash' was used because there was one message -there with 8 bit headers which dbmail doesn't accept, so I had to skip -the whole folder. It would be nice to have an option to just ignore -and log unsyncable messages, but do the rest, instead of stopping. - -****************** - -There are two other major problems: - -1) dbmail doesn't accept utf8 header, while cyrus does. imapsync stops -in that case, making sync impossible - -To convert the whole messages from 8bit to 7bit, use option : - ---regexmess 's/[\x80-\xff]/X/g' - -(This is bad since only headers need this). +Default behavior might works. ====================================================================== Q: From MailEnable 1.75 @@ -1264,9 +1209,8 @@ R. Use: ====================================================================== Q. From Courier to Archiveopteryx -R. http://www.archiveopteryx.org/migration/imapsync -Use: - --useheader Message-Id --skipsize +R. You can read http://www.archiveopteryx.org/migration/imapsync + Default values might be fine now with latest imapsync. ====================================================================== Q. To Sun Java(tm) System Messaging Server 6.2-7.05 @@ -1297,8 +1241,7 @@ expensive nonfree software). Q. From Microsoft Exchange 2000 IMAP4rev1 server version 6.0.6487.0. R. imapsync ... \ - --prefix1 INBOX. --prefix2 INBOX. --syncinternaldates --subscribe \ - --maxsize 10485760 + --prefix1 INBOX. ====================================================================== Q: How can I write an .rpm with imapsync diff --git a/INSTALL b/INSTALL index 87d6284..9969b13 100644 --- a/INSTALL +++ b/INSTALL @@ -1,4 +1,4 @@ -# $Id: INSTALL,v 1.20 2011/05/07 02:14:58 gilles Exp gilles $ +# $Id: INSTALL,v 1.21 2011/11/17 08:07:03 gilles Exp gilles $ # # INSTALL file for imapsync # imapsync : IMAP sync or copy tool. @@ -6,15 +6,15 @@ INTRODUCTION ------------ - imapsync works fine under any Unix OS with perl. - imapsync.exe works fine under Windows XP, Vista, Seven, 20XX. + imapsync works fine under any operating system with Perl and Perl modules (listed below). + imapsync.exe works fine standalone under Windows XP, Vista, Seven, 20XX, either 32 or 64bit. UNIX ---- - Buy imapsync at - http://www.linux-france.org/prj/imapsync/ + Purchase imapsync at + http://imapsync.lamiral.info/ You'll have access to a compressed tarball called imapsync-x.xx.tgz where x.xx is the version number. Untar the tarball where you want (on Unix): @@ -27,11 +27,10 @@ UNIX WINDOWS ------- - a) Simplest way: -- Buy imapsync.exe at http://www.linux-france.org/prj/imapsync/ -- Use imapsync.exe. +- Buy imapsync.exe at http://imapsync.lamiral.info/ +- Run imapsync.exe in a command prompt (execute cmd.exe). b) Hard way: @@ -42,7 +41,6 @@ b) Hard way: PPM is Perl Package Manager. - PREREQUISITES ------------- diff --git a/README b/README index f3df037..ea85932 100644 --- a/README +++ b/README @@ -3,7 +3,7 @@ NAME Synchronise mailboxes between two imap servers. Good at IMAP migration. More than 36 different IMAP server softwares supported with success. - $Revision: 1.463 $ + $Revision: 1.468 $ SYNOPSIS To synchronise imap account "foo" on "imap.truc.org" to imap account @@ -20,12 +20,12 @@ INSTALL with Strawberry Perl 5.10 or 5.12 or as a standalone binary software imapsync.exe - imapsync is already available directly on the following distributions - (at least): FreeBSD, Debian, Ubuntu, Gentoo, Fedora, NetBSD, Darwin, - Mandriva and OpenBSD. + imapsync can be available directly on the following distributions: + FreeBSD, Debian, Ubuntu, Gentoo, Fedora, NetBSD, Darwin, Mandriva and + OpenBSD. - Get imapsync at - http://www.linux-france.org/prj/imapsync/ + Purchase latest imapsync at + http://imapsync.lamiral.info/ You'll receive a link to a compressed tarball called imapsync-x.xx.tgz where x.xx is the version number. Untar the tarball where @@ -35,9 +35,10 @@ INSTALL Go into the directory imapsync-x.xx and read the INSTALL file. The INSTALL file is also at - http://www.linux-france.org/prj/imapsync/INSTALL + http://imapsync.lamiral.info/INSTALL - The freshmeat record is at http://freshmeat.net/projects/imapsync/ + The freecode (was freshmeat) record is at + http://freecode.com/projects/imapsync USAGE imapsync [options] @@ -238,7 +239,7 @@ BUG REPORT GUIDELINES author. Before reporting bugs, read the FAQ, the README and the TODO files. - http://www.linux-france.org/prj/imapsync/ + http://imapsync.lamiral.info/ Upgrade to last imapsync release, maybe the bug is already fixed. @@ -442,5 +443,5 @@ SIMILAR SOFTWARES Feedback (good or bad) will often be welcome. - $Id: imapsync,v 1.463 2011/11/12 21:58:52 gilles Exp gilles $ + $Id: imapsync,v 1.468 2011/11/18 01:23:37 gilles Exp gilles $ diff --git a/TODO b/TODO index 6295f99..b7e4207 100644 --- a/TODO +++ b/TODO @@ -1,5 +1,5 @@ #!/bin/cat -# $Id: TODO,v 1.103 2011/11/12 21:59:35 gilles Exp gilles $ +# $Id: TODO,v 1.104 2011/11/17 15:45:57 gilles Exp gilles $ TODO file for imapsync ---------------------- @@ -48,13 +48,6 @@ e-mail when it gets synced, rather than just the message ID and the date/time. Add a well described problem for each problem detected and counted in error counter statistics. -Add yahoo imap support: -http://elearningcentral.blogspot.com/2010/07/how-to-use-imapsync-with-yahoo-imap.html -http://www.bwebcentral.com/utils/imapsync-yahoo -See patches/imapsync-yahoo - - - Add "output to reflect everything that imapsync was doing". Not everything but flag synchronization will be nice" @@ -75,11 +68,8 @@ Peer Heinlein. Add different levels of output to see clearly the problem by default. -Add option --exclude_messages_with_flag - Add more information about skipped messages. - Add Rick Romero patch with --quiet No output at all --showstats intended for use with --quiet @@ -90,7 +80,6 @@ issues so far". Sounds good! Add an option to implement the faq entry about copying a contact folder. - imapsync doesn't report well. It should says "I had to sync 123 messages but I could transfer only 99 messages" Maybe count messages not transfered because they're dupplicate. @@ -163,7 +152,18 @@ http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html =========================================================================== -DONE. Add --subscribe by default. +DONE. The --search option allows this (and many other things) +Add option --exclude_messages_with_flag + +DONE. (yahoo login is standard now) +Add yahoo imap support: +http://elearningcentral.blogspot.com/2010/07/how-to-use-imapsync-with-yahoo-imap.html +http://www.bwebcentral.com/utils/imapsync-yahoo +See patches/imapsync-yahoo + + +DONE. Add --subscribe by default. +Subscribe only when needed (not already subscribed). DONE. Ask Nick Czeczulin why he wrote patent "Method for mailbox migration" http://www.google.com/patents/about?id=qI_IAAAAEBAJ diff --git a/VERSION b/VERSION index 5b29492..11d932e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.463 +1.468 diff --git a/VERSION_EXE b/VERSION_EXE index 5679d30..11d932e 100644 --- a/VERSION_EXE +++ b/VERSION_EXE @@ -1 +1 @@ -1.462 +1.468 diff --git a/W/paypal_reply/paypal_build_invoices b/W/paypal_reply/paypal_build_invoices index b7548ad..f999034 100755 --- a/W/paypal_reply/paypal_build_invoices +++ b/W/paypal_reply/paypal_build_invoices @@ -16,12 +16,14 @@ set -x #/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 733 /g/paypal/paypal_2011_06_complet.csv #/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 824 /g/paypal/paypal_2011_07_complet.csv #/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 917 /g/paypal/paypal_2011_08_complet.csv -/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 999 /g/paypal/paypal_2011_09_complet.csv -/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 1094 /g/paypal/paypal_2011_10_complet.csv +#/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 999 /g/paypal/paypal_2011_09_complet.csv +#/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 1094 /g/paypal/paypal_2011_10_complet.csv +/g/public_html/imapsync/W/paypal_reply/paypal_bilan --write_invoices --first_in 1185 /g/paypal/paypal_2011_11_complet.csv : /g/public_html/imapsync/W/paypal_reply/paypal_bilan --first_in 824 /g/paypal/paypal_2011_07_complet.csv : /g/public_html/imapsync/W/paypal_reply/paypal_bilan --first_in 917 /g/paypal/paypal_2011_08_complet.csv : /g/public_html/imapsync/W/paypal_reply/paypal_bilan --first_in 1094 /g/paypal/paypal_2011_10_complet.csv +: /g/public_html/imapsync/W/paypal_reply/paypal_bilan --first_in 1185 /g/paypal/paypal_2011_11_complet.csv set +x # La totale @@ -34,7 +36,6 @@ set +x /g/paypal/paypal_201?_??_complet.csv - # USD de 147 à 340 # EUR de 341 à ... diff --git a/imapsync b/imapsync index e8a1215..6d2caea 100755 --- a/imapsync +++ b/imapsync @@ -20,7 +20,7 @@ Synchronise mailboxes between two imap servers. Good at IMAP migration. More than 36 different IMAP server softwares supported with success. -$Revision: 1.463 $ +$Revision: 1.468 $ =head1 SYNOPSIS @@ -40,13 +40,12 @@ To synchronise imap account "foo" on "imap.truc.org" with Strawberry Perl 5.10 or 5.12 or as a standalone binary software imapsync.exe -imapsync is already available directly on the following distributions -(at least): +imapsync can be available directly on the following distributions: FreeBSD, Debian, Ubuntu, Gentoo, Fedora, NetBSD, Darwin, Mandriva and OpenBSD. - Get imapsync at - http://www.linux-france.org/prj/imapsync/ + Purchase latest imapsync at + http://imapsync.lamiral.info/ You'll receive a link to a compressed tarball called imapsync-x.xx.tgz where x.xx is the version number. Untar the tarball where @@ -56,9 +55,10 @@ NetBSD, Darwin, Mandriva and OpenBSD. Go into the directory imapsync-x.xx and read the INSTALL file. The INSTALL file is also at - http://www.linux-france.org/prj/imapsync/INSTALL + http://imapsync.lamiral.info/INSTALL - The freshmeat record is at http://freshmeat.net/projects/imapsync/ + The freecode (was freshmeat) record is at + http://freecode.com/projects/imapsync =head1 USAGE @@ -285,7 +285,7 @@ Report any bugs or feature requests to the public mailing-list or to the author. Before reporting bugs, read the FAQ, the README and the -TODO files. http://www.linux-france.org/prj/imapsync/ +TODO files. http://imapsync.lamiral.info/ Upgrade to last imapsync release, maybe the bug is already fixed. @@ -517,7 +517,7 @@ Entries for imapsync: Feedback (good or bad) will often be welcome. -$Id: imapsync,v 1.463 2011/11/12 21:58:52 gilles Exp gilles $ +$Id: imapsync,v 1.468 2011/11/18 01:23:37 gilles Exp gilles $ =cut @@ -631,7 +631,7 @@ my( # global variables initialisation -$rcs = '$Id: imapsync,v 1.463 2011/11/12 21:58:52 gilles Exp gilles $ '; +$rcs = '$Id: imapsync,v 1.468 2011/11/18 01:23:37 gilles Exp gilles $ '; $total_bytes_transferred = 0; $total_bytes_skipped = 0; @@ -926,14 +926,17 @@ exit_clean(0) if ($justlogin); # my ( -@h1_folders_all, %h1_folders_all, @h1_folders_wanted, %requested_folder, %subscribed_folder, +@h1_folders_all, %h1_folders_all, @h1_folders_wanted, %requested_folder, +%h1_subscribed_folder, %h2_subscribed_folder, @h2_folders_all, %h2_folders_all, @h2_folders_from_1_wanted, %h2_folders_from_1_wanted, @h2_folders_from_1_all, %h2_folders_from_1_all, ); -# Make a hash of subscribed folders in source server. -map { $subscribed_folder{$_} = 1 } $imap1->subscribed(); +# Make a hash of subscribed folders in both servers. +map { $h1_subscribed_folder{ $_ } = 1 } $imap1->subscribed( ); +map { $h2_subscribed_folder{ $_ } = 1 } $imap2->subscribed( ); + # All folders on host1 and host2 @h1_folders_all = sort $imap1->folders(); @@ -949,8 +952,8 @@ if (scalar(@folder) or $subscribed or scalar(@folderrec)) { } # option --subscribed - if ($subscribed) { - add_to_requested_folders(keys (%subscribed_folder)); + if ( $subscribed ) { + add_to_requested_folders( keys ( %h1_subscribed_folder ) ) ; } # option --folderrec @@ -1053,8 +1056,8 @@ print print "Host1 subscribed folders list: ", - map("[$_] ", sort keys(%subscribed_folder)), "\n" - if ($subscribed); + map( "[$_] ", sort keys( %h1_subscribed_folder ) ), "\n" + if ( $subscribed ) ; my @h2_folders_not_in_1; @h2_folders_not_in_1 = list_folders_in_2_not_in_1(); @@ -1069,35 +1072,42 @@ print "++++ Looping on each folder\n"; FOLDER: foreach my $h1_fold (@h1_folders_wanted) { - my $h2_fold = imap2_folder_name($h1_fold); + my $h2_fold = imap2_folder_name( $h1_fold ) ; #relogin1( ) if ( $relogin1 ) ; - printf("%-35s -> %-35s\n", "[$h1_fold]", "[$h2_fold]"); + printf( "%-35s -> %-35s\n", "[$h1_fold]", "[$h2_fold]" ) ; - select_folder($imap1, $h1_fold, 'Host1') or next FOLDER; + # host1 can be fetched read only, select is not needed. + examine_folder( $imap1, $h1_fold, 'Host1' ) or next FOLDER ; - if ( ! exists($h2_folders_all{$h2_fold})) { - create_folder( $imap2, $h2_fold, $h1_fold ) or next FOLDER; + if ( ! exists( $h2_folders_all{ $h2_fold } ) ) { + create_folder( $imap2, $h2_fold, $h1_fold ) or next FOLDER ; } - acls_sync($h1_fold, $h2_fold); + acls_sync( $h1_fold, $h2_fold ) ; - select_folder($imap2, $h2_fold, 'Host2') or next FOLDER; - my @select_results = $imap2->Results(); + # Sometimes the folder on host2 is listed (it exists) but is + # not selectable but becomes selectable by a create (Gmail) + select_folder( $imap2, $h2_fold, 'Host2' ) + or ( create_folder( $imap2, $h2_fold, $h1_fold ) + and select_folder( $imap2, $h2_fold, 'Host2' ) ) + or next FOLDER ; + my @select_results = $imap2->Results( ) ; - #print "%%% @select_results\n"; - my $permanentflags2 = permanentflags(@select_results); + #print "%%% @select_results\n" ; + my $permanentflags2 = permanentflags( @select_results ) ; $debug and print "permanentflags: $permanentflags2\n" ; if ( $expunge or $expunge1 ){ - print "Expunging host1 $h1_fold\n"; - unless($dry) { $imap1->expunge() }; - #print "Expunging host2 $h2_fold\n"; - #unless($dry) { $imap2->expunge() }; + print "Expunging host1 $h1_fold\n" ; + unless($dry) { $imap1->expunge() } ; + #print "Expunging host2 $h2_fold\n" ; + #unless($dry) { $imap2->expunge() } ; } - if (($subscribe and exists $subscribed_folder{$h1_fold}) or $subscribe_all) { - print "Subscribing to folder $h2_fold on destination server\n"; - unless($dry) { $imap2->subscribe($h2_fold) }; + if ( ( ( $subscribe and exists $h1_subscribed_folder{ $h1_fold } ) or $subscribe_all ) + and ! exists $h2_subscribed_folder{ $h2_fold } ) { + print "Subscribing to folder $h2_fold on destination server\n" ; + unless( $dry ) { $imap2->subscribe( $h2_fold ) } ; } next FOLDER if ($justfolders); @@ -1173,7 +1183,7 @@ FOLDER: foreach my $h1_fold (@h1_folders_wanted) { @$h1_fir_ref{@h1_msgs} = (undef); $debug and print "Host1 getting flags idate and sizes of folder [$h1_fold]\n" ; - $h1_fir_ref = $imap1->fetch_hash_2("FLAGS", "INTERNALDATE", "RFC822.SIZE", $h1_fir_ref) + $h1_fir_ref = $imap1->fetch_hash("FLAGS", "INTERNALDATE", "RFC822.SIZE", $h1_fir_ref) if (@h1_msgs); $debug and print "Host1 getting flags idate and sizes of folder [$h1_fold] took ", timenext(), " s\n"; unless ($h1_fir_ref) { @@ -1214,7 +1224,7 @@ FOLDER: foreach my $h1_fold (@h1_folders_wanted) { $debug and print "Host2 getting flags idate and sizes of folder [$h2_fold]\n" ; @$h2_fir_ref{@h2_msgs} = ( ); # fetch_hash_2 can select by uid with last arg as ref - $h2_fir_ref = $imap2->fetch_hash_2("FLAGS", "INTERNALDATE", "RFC822.SIZE", $h2_fir_ref) + $h2_fir_ref = $imap2->fetch_hash("FLAGS", "INTERNALDATE", "RFC822.SIZE", $h2_fir_ref) if (@h2_msgs); $debug and print "Host2 getting flags idate and sizes of folder [$h2_fold] took ", timenext(), " s\n" ; @@ -1827,8 +1837,8 @@ sub banner_imapsync { my @argv_copy = @_; my $banner_imapsync = join("", '$RCSfile: imapsync,v $ ', - '$Revision: 1.463 $ ', - '$Date: 2011/11/12 21:58:52 $ ', + '$Revision: 1.468 $ ', + '$Date: 2011/11/18 01:23:37 $ ', "\n",localhost_info(), "\n", "Command line used:\n", "$0 ", command_line_nopassword(@argv_copy), "\n", @@ -1882,16 +1892,30 @@ sub missing_option { sub select_folder { - my ($imap, $folder, $hostside) = @_; - if ( ! $imap->select($folder)) { + my ( $imap, $folder, $hostside ) = @_ ; + if ( ! $imap->select( $folder ) ) { print "$hostside folder $folder: Could not select: ", - $imap->LastError, "\n"; - $nb_errors++; - return(0); + $imap->LastError, "\n" ; + $nb_errors++ ; + return( 0 ) ; }else{ # ok select succeeded - return(1); + return( 1 ) ; + } +} + +sub examine_folder { + my ( $imap, $folder, $hostside ) = @_ ; + if ( ! $imap->examine( $folder ) ) { + print + "$hostside folder $folder: Could not examine: ", + $imap->LastError, "\n" ; + $nb_errors++ ; + return( 0 ) ; + }else{ + # ok examine succeeded + return( 1 ) ; } } @@ -2216,56 +2240,56 @@ sub imap2_folder_name { sub foldersizes { - my ($side, $imap, @folders) = @_; - my $total_size = 0; - my $total_nb = 0; + my ( $side, $imap, @folders ) = @_ ; + my $total_size = 0 ; + my $total_nb = 0 ; my $biggest = 0 ; - print "++++ Calculating sizes\n"; - foreach my $folder (@folders) { - my $stot = 0; - my $nb_msgs = 0; - printf("$side folder %-35s", "[$folder]"); + print "++++ Calculating sizes\n" ; + foreach my $folder ( @folders ) { + my $stot = 0 ; + my $nb_msgs = 0 ; + printf( "$side folder %-35s", "[$folder]" ) ; if ( 'Host2' eq $side and ! exists( $h2_folders_all{ $folder } ) ) { print(" does not exist yet\n") ; next ; } if ( 'Host1' eq $side and ! exists( $h1_folders_all{ $folder } ) ) { - print(" does not exist\n") ; + print( " does not exist\n" ) ; next ; } - unless ($imap->examine($folder)) { + unless ( $imap->examine( $folder ) ) { print "$side Folder $folder: Could not examine: ", - $imap->LastError, "\n"; - $nb_errors++; - next; + $imap->LastError, "\n" ; + $nb_errors++ ; + next ; } - my $hash_ref = {}; - my @msgs = select_msgs($imap); - $nb_msgs = scalar(@msgs); + my $hash_ref = { } ; + my @msgs = select_msgs( $imap ) ; + $nb_msgs = scalar( @msgs ) ; my $smax = 0 ; - @$hash_ref{@msgs} = (undef) if @msgs ; + @$hash_ref{ @msgs } = ( undef ) if @msgs ; if ( $nb_msgs > 0 and @msgs ) { - $imap->fetch_hash_2("RFC822.SIZE",$hash_ref) or die_clean("$@"); - #print map {$hash_ref->{$_}->{"RFC822.SIZE"}, " "} keys %$hash_ref; - map {$stot += $hash_ref->{$_}->{"RFC822.SIZE"}} keys %$hash_ref ; - $smax = max( map {$hash_ref->{$_}->{"RFC822.SIZE"}} keys %$hash_ref ); - $biggest = max( $biggest, $smax ); + $imap->fetch_hash("RFC822.SIZE",$hash_ref) or die_clean("$@") ; + #print map {$hash_ref->{$_}->{"RFC822.SIZE"}, " "} keys %$hash_ref ; + map { $stot += $hash_ref->{ $_ }->{ "RFC822.SIZE" } } keys %$hash_ref ; + $smax = max( map { $hash_ref->{ $_ }->{ "RFC822.SIZE" } } keys %$hash_ref ) ; + $biggest = max( $biggest, $smax ) ; } - printf(" Size: %9s", $stot); - printf(" Messages: %5s", $nb_msgs); - printf(" Biggest: %9s\n", $smax); - $total_size += $stot; - $total_nb += $nb_msgs; + printf( " Size: %9s", $stot ) ; + printf( " Messages: %5s", $nb_msgs ) ; + printf( " Biggest: %9s\n", $smax ) ; + $total_size += $stot ; + $total_nb += $nb_msgs ; } - printf ("Nb messages: %11s\n", $total_nb ) ; - printf ("Total size: %11s bytes\n", $total_size ) ; - printf ("Biggest message: %11s bytes\n", $biggest ) ; - printf ("Time: %11s secondes\n", timenext( ) ) ; + printf ( "Nb messages: %11s\n", $total_nb ) ; + printf ( "Total size: %11s bytes\n", $total_size ) ; + printf ( "Biggest message: %11s bytes\n", $biggest ) ; + printf ( "Time: %11s secondes\n", timenext( ) ) ; } sub timenext { @@ -2736,8 +2760,8 @@ sub cache_map { $debugcache and print "C12: $file\n" ; ( $uid1, $uid2 ) = match_a_cache_file( $file ) ; - if ( exists( $h1_msgs_hash_ref->{ $uid1 // q{} } ) - and exists( $h2_msgs_hash_ref->{ $uid2 // q{} } ) ) { + if ( exists( $h1_msgs_hash_ref->{ defined( $uid1 ) ? $uid1 : q{} } ) + and exists( $h2_msgs_hash_ref->{ defined( $uid2 ) ? $uid2 : q{} } ) ) { # keep only the greatest uid2 # 130_2301 and # 130_231 => keep only 130 -> 2301 @@ -3276,17 +3300,13 @@ sub stats { sub thank_author { - return("Homepage: http://www.linux-france.org/prj/imapsync/\n"); + return("Homepage: http://imapsync.lamiral.info/\n"); - my $basename = imapsync_basename(); - $debug and print "[$basename]\n"; - return("Homepage: http://www.linux-france.org/prj/imapsync/\n") - if ( $basename =~ /\.exe$|\.bin$/ ); - + # used to be return(join("", "Happy with this free, open and gratis DWTFPL software?\n", "Encourage the author (Gilles LAMIRAL) by giving him a book\n", "or just money via paypal:\n", - "http://www.linux-france.org/prj/imapsync/\n")); + "http://imapsync.lamiral.info/\n")); } sub get_options { @@ -3586,7 +3606,7 @@ sub check_last_release { } sub imapsync_version { - my $rcs = '$Id: imapsync,v 1.463 2011/11/12 21:58:52 gilles Exp gilles $ '; + my $rcs = '$Id: imapsync,v 1.468 2011/11/18 01:23:37 gilles Exp gilles $ '; $rcs =~ m/,v (\d+\.\d+)/; my $VERSION = ($1) ? $1: "UNKNOWN"; return($VERSION); @@ -3620,7 +3640,7 @@ sub imapsync_version_lfo { print $sock "GET /prj/imapsync/VERSION HTTP/1.0\n", "User-Agent: imapsync/$local_version ($agent_info)\n", - "Host: www.linux-france.org\n\n"; + "Host: ks.lamiral.info\n\n"; my @line = <$sock>; close($sock); my $last_release = $line[-1]; @@ -4477,18 +4497,30 @@ use constant NonFolderArg => 1; # Value to pass to Massage to }; - *Mail::IMAPClient::fetch_hash = sub { - # taken from original lib, - # just added split code. + # taken from above *Mail::IMAPClient::fetch_hash + # if last arg is a ref then the fetch is done only + # on the messages listed as the keys of this hash. + # Init an "empty" $hash_ref by value can be done this way: + # @$hash_ref{2, 3, 4, 55} = (undef); + my $self = shift; - my $hash = ref($_[-1]) ? pop @_ : {}; + my $hash_ref = ref($_[-1]) ? pop @_ : {}; my @words = @_; for (@words) { s/([\( ])FAST([\) ])/${1}FLAGS INTERNALDATE RFC822\.SIZE$2/i ; s/([\( ])FULL([\) ])/${1}FLAGS INTERNALDATE RFC822\.SIZE ENVELOPE BODY$2/i ; } - my $msgs_ref_all = scalar($self->messages); + + my $msgs_ref_all; + if (scalar %$hash_ref) { + $msgs_ref_all = [ sort { $a <=> $b } keys (%$hash_ref) ]; + #print "ZZZZ 1 [@$msgs_ref_all]\n"; + }else{ + $msgs_ref_all = scalar($self->messages); + #print "ZZZZ 2 [@$msgs_ref_all]\n"; + } + my $split = $self->Split() || scalar(@$msgs_ref_all); while(my @msgs = splice(@$msgs_ref_all, 0, $split)) { #print "SPLIT: @msgs\n"; @@ -4502,21 +4534,21 @@ use constant NonFolderArg => 1; # Value to pass to Massage to if ($self->Uid) { my($uid) = $l =~ /\((?:.* )?UID (\d+).*\)/i; next unless $uid; - if ( exists $hash->{$uid} ) { - $entry = $hash->{$uid} ; + if ( defined $hash_ref->{$uid} ) { + $entry = $hash_ref->{$uid} ; } else { - $hash->{$uid} ||= $entry; + $hash_ref->{$uid} ||= $entry; } } else { my($mid) = $l =~ /^\* (\d+) FETCH/i; next unless $mid; - if ( exists $hash->{$mid} ) { - $entry = $hash->{$mid} ; + if ( defined $hash_ref->{$mid} ) { + $entry = $hash_ref->{$mid} ; } else { - $hash->{$mid} ||= $entry; + $hash_ref->{$mid} ||= $entry; } } @@ -4547,7 +4579,7 @@ use constant NonFolderArg => 1; # Value to pass to Massage to } } } - return wantarray ? %$hash : $hash; + return wantarray ? %$hash_ref : $hash_ref; }; @@ -5418,87 +5450,3 @@ sub capability_update { $self->capability; } -sub fetch_hash_2 { - # taken from above *Mail::IMAPClient::fetch_hash - # if last arg is a ref then the fetch is done only - # on the messages listed as the keys of this hash. - # Init an "empty" $hash_ref by value can be done this way: - # @$hash_ref{2, 3, 4, 55} = (undef); - - my $self = shift; - my $hash_ref = ref($_[-1]) ? pop @_ : {}; - my @words = @_; - for (@words) { - s/([\( ])FAST([\) ])/${1}FLAGS INTERNALDATE RFC822\.SIZE$2/i ; - s/([\( ])FULL([\) ])/${1}FLAGS INTERNALDATE RFC822\.SIZE ENVELOPE BODY$2/i ; - } - - my $msgs_ref_all; - if (scalar %$hash_ref) { - $msgs_ref_all = [ sort { $a <=> $b } keys (%$hash_ref) ]; - #print "ZZZZ 1 [@$msgs_ref_all]\n"; - }else{ - $msgs_ref_all = scalar($self->messages); - #print "ZZZZ 2 [@$msgs_ref_all]\n"; - } - - my $split = $self->Split() || scalar(@$msgs_ref_all); - while(my @msgs = splice(@$msgs_ref_all, 0, $split)) { - #print "SPLIT: @msgs\n"; - my $msgs_ref = \@msgs; - my $output = scalar($self->fetch($msgs_ref,"(" . join(" ",@_) . ")")) - ; # unless grep(/\b(?:FAST|FULL)\b/i,@words); - my $x; - for ($x = 0; $x <= $#$output ; $x++) { - my $entry = {}; - my $l = $output->[$x]; - if ($self->Uid) { - my($uid) = $l =~ /\((?:.* )?UID (\d+).*\)/i; - next unless $uid; - if ( defined $hash_ref->{$uid} ) { - $entry = $hash_ref->{$uid} ; - } - else { - $hash_ref->{$uid} ||= $entry; - } - } - else { - my($mid) = $l =~ /^\* (\d+) FETCH/i; - next unless $mid; - if ( defined $hash_ref->{$mid} ) { - $entry = $hash_ref->{$mid} ; - } - else { - $hash_ref->{$mid} ||= $entry; - } - } - - foreach my $w (@words) { - if ( $l =~ /\Q$w\E\s*$/i ) { - $entry->{$w} = $output->[$x+1]; - $entry->{$w} =~ s/(?:\x0a?\x0d)+$//g; - chomp $entry->{$w}; - } - else { - $l =~ /\( # open paren followed by ... - (?:.*\s)? # ...optional stuff and a space - \Q$w\E\s # escaped fetch field - (?:" # then: a dbl-quote - (\\.| # then bslashed anychar(s) or ... - [^"]+) # ... nonquote char(s) - "| # then closing quote; or ... - \( # ...an open paren - (\\.| # then bslashed anychar or ... - [^\)]*) # ... non-close-paren char - \)| # then closing paren; or ... - (\S+)) # unquoted string - (?:\s.*)? # possibly followed by space-stuff - \) # close paren - /xi; - $entry->{$w}=defined($1)?$1:defined($2)?$2:$3; - } - } - } -} - return wantarray ? %$hash_ref : $hash_ref; -} diff --git a/index.shtml b/index.shtml index e0918a6..cb2253b 100644 --- a/index.shtml +++ b/index.shtml @@ -5,7 +5,7 @@ Imapsync: an IMAP migration tool ( release <!--#exec cmd="cat VERSION"--> ) - + @@ -36,7 +36,7 @@
-imapsync logo +imapsync logo

Welcome to the imapsync web site!

@@ -86,14 +86,22 @@ where the user plays independently on both sides. Use offlineimap

New features or bugfixes since previous releases:

+ +