diff --git a/ChangeLog b/ChangeLog index b97345d..f55f08c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,17 +1,164 @@ RCS file: RCS/imapsync,v Working file: imapsync -head: 1.921 +head: 1.945 branch: locks: strict - gilles: 1.921 + gilles: 1.945 access list: symbolic names: keyword substitution: kv -total revisions: 921; selected revisions: 921 +total revisions: 945; selected revisions: 945 description: ---------------------------- -revision 1.921 locked by: gilles; +revision 1.945 locked by: gilles; +date: 2019/06/26 19:30:56; author: gilles; state: Exp; lines: +151 -94 +Proofread the documentation part, what will go to README. +The pod at the beginning. +---------------------------- +revision 1.944 +date: 2019/06/25 16:44:08; author: gilles; state: Exp; lines: +8 -8 +Bugfix. Readonly syntax. "Readonly my %EXIT_TXT => (" +---------------------------- +revision 1.943 +date: 2019/06/25 08:34:51; author: gilles; state: Exp; lines: +47 -30 +Usability. Added the error message text after "Exiting with return value xx". Example: Exiting with return value 16 (EXIT_AUTHENTICATION_FAILURE) +Usability. Added what function is called when installing a signal. Example: "kill -QUIT 19839 # special behavior: call to sub catch_exit" +---------------------------- +revision 1.942 +date: 2019/06/24 20:43:43; author: gilles; state: Exp; lines: +51 -37 +Bugfix. USR1 signal for toggle_sleep call was never installed. +Usability. Now --justfolders will activate --foldersizesatend +Enhancement. Added PROCESS_ID when "Tail -f finished." +Bugfix. The number of messages deleted was not accurate at all. +Usability. masked the warning "no header by parse_headers so taking whole header with BODY.PEEK[HEADER]" to --debug mode. +Usability. Moved the warning "no header found so adding our own" to --debug mode. +Tests. Added ETA simulation with --simulong (for the progress info on /X) +Bugfix. Change the longest line in --exchange2 and --office from 10500 to 10239. (\n inserted for those long lines) +Enhancement. --maxage and --minage can take float as value in order to filter with less than a day. (Need --noabletosearch also). +---------------------------- +revision 1.941 +date: 2019/06/04 22:55:24; author: gilles; state: Exp; lines: +54 -19 +Usability. Now "imapsync --host1 ks" or "imapsync --host2" is ok and do like a --justconnect on servers. Useful to cjeck rapidely a imap server. +---------------------------- +revision 1.940 +date: 2019/06/04 06:29:31; author: gilles; state: Exp; lines: +14 -9 +Added 'junk', 'Junk E-Mail', 'Junk Email' as possible_special of \Junk +Added 'Deleted Messages' as possible_special of \Trash +---------------------------- +revision 1.939 +date: 2019/05/27 21:58:33; author: gilles; state: Exp; lines: +10 -9 +Back to a test Mail::IMAPClient 3.40 failed but Mail::IMAPClient 3.42 succeed. +---------------------------- +revision 1.938 +date: 2019/05/27 18:45:06; author: gilles; state: Exp; lines: +111 -33 +CGI context. --addheader is on by default. +Added option --appendlimit xxxx in order to override the value got by APPENDLIMIT in CAPABILITY. +Added option --truncmess to allow transfer of too big messages. When you think it's better to have a truncated message than no message at all. +1453 regression tests. +---------------------------- +revision 1.937 +date: 2019/05/01 22:14:00; author: gilles; state: Exp; lines: +328 -326 +CGI context. Only Message-Id to identify messages, instead of Message-Id Received. +Fixed perl critic about blanks. +Fixed tests comments +---------------------------- +revision 1.936 +date: 2019/04/29 05:26:19; author: gilles; state: Exp; lines: +26 -23 +Bugfix. Do not tail when abort. +Usability. --testslive values can be overwritten like --testslive --user1 empty + Same thing for --testslive6 +---------------------------- +revision 1.935 +date: 2019/04/28 02:10:24; author: gilles; state: Exp; lines: +293 -44 +Added File::Tail dependency. +Added --tail option. Will do like a tail -f on the logfile of another imapsync with the same pidfile. + Good in CGI context. Maybe... + We'll see. It was one of my first whishes for cgi behaviour +1441 unit tests. +---------------------------- +revision 1.934 +date: 2019/04/26 08:23:24; author: gilles; state: Exp; lines: +134 -52 +Added following options available in cgi context: +--subfolder1 +--subfolder2 +--justfolders +--justfoldersizes +--delete1 +--delete2 + +CGI context. Added the equivalent command line parameters like: +Command line used, run by /usr/bin/perl: +/usr/lib/cgi-bin/imapsync --host1 test1.lamiral.info +Usability. Turns off --automap when --subfolder1 or --subfolder2 is used. +---------------------------- +revision 1.933 +date: 2019/04/23 11:42:42; author: gilles; state: Exp; lines: +47 -98 +CGI context. Added default like --maxsize 1_000_000_000 (Out of memory prevention?) +Usability. maxsize is the minimum of --maxsize and APPENDLIMIT=xxx or each one if the other no exists. +---------------------------- +revision 1.932 +date: 2019/04/13 22:09:04; author: gilles; state: Exp; lines: +93 -24 +Added --sanitize to remove egding blanks on host1 user1 password1 etc. +--sanitize on by default. Use --noanitize to avoid it +---------------------------- +revision 1.931 +date: 2019/04/13 16:10:24; author: gilles; state: Exp; lines: +109 -22 +Added test tests_check_binary_embed_all_dyn_libs to detect non embedded dynamic libraries in binaries Darwin and Win32 +---------------------------- +revision 1.930 +date: 2019/04/11 22:32:38; author: gilles; state: Exp; lines: +467 -211 +All exits are now done with a specific return code related to the error causing the exit. +Removed sub die_clean() +die_clean() calls replaced by exit_clean() calls +---------------------------- +revision 1.929 +date: 2019/04/08 09:14:13; author: gilles; state: Exp; lines: +425 -300 +Bug fix. With --gmail1 or --gmail2 do not sync [Gmail]/All Mail if it is not requested. +Bug fix. When asked to terminate by a signal, kill myself by the signal instead of an exit call. + Applied Martin Cracauer's advice on https://www.cons.org/cracauer/sigint.html + "Proper handling of SIGINT/SIGQUIT" + I hope it fixes the not always working double-ctrl-c to end the sync. + See bug report https://github.com/imapsync/imapsync/issues/142 +---------------------------- +revision 1.928 +date: 2019/03/28 11:12:44; author: gilles; state: Exp; lines: +648 -317 +Added --subfolder1 Foo/Bar option in order to restore a backup made with --subfolder2 Foo/Bar + --folderrec Foo/Bar is imply by --subfolder1 Foo/Bar because it's the most common purpose. +Deglobalized $debug @regextrans2 $sep1 $sep2 $fixslash2 +Added --resynclabels to resync the Gmail labels X-GM-LABELS +Bugfix. --synclabels now create sublabels with --subfolder2 +Bugfix. --subfolder2 Pf2/Sub2 and --subfolder2 Sub2 perform the same when Pf2 is the --prefix2 +1336 non-regression tests +---------------------------- +revision 1.927 +date: 2019/03/22 14:45:55; author: gilles; state: Exp; lines: +9 -8 +Active --resynclabels when --gmail1 --gmail2 +---------------------------- +revision 1.926 +date: 2019/03/22 14:06:38; author: gilles; state: Exp; lines: +283 -88 +Call abort() before simulong() to be able to abort a long simulong. +Bugfix. --synclabels --subfolder2 create sublabels only and not a mixture. +Enhancement. Added --resynclabels +1286 non-regression tests +---------------------------- +revision 1.925 +date: 2019/03/03 12:25:02; author: gilles; state: Exp; lines: +8 -8 +1268 tests +---------------------------- +revision 1.924 +date: 2019/03/03 12:22:37; author: gilles; state: Exp; lines: +50 -27 +Tolerate more load. Double the threshold. accept load at 2 per core instead of 1 per core +---------------------------- +revision 1.923 +date: 2019/03/03 12:00:05; author: gilles; state: Exp; lines: +29 -20 +Moved cgiload( $sync ) in order to log exites on heavy load +---------------------------- +revision 1.922 +date: 2019/02/26 13:03:58; author: gilles; state: Exp; lines: +43 -35 +Fixed tests to pass under the docker image +---------------------------- +revision 1.921 date: 2019/02/18 10:21:03; author: gilles; state: Exp; lines: +10 -8 Bugfix. Errors of APPEND were not counted as missing messages on host2! ---------------------------- @@ -192,7 +339,7 @@ Default $sync->{ sigignore } to empty list. ---------------------------- revision 1.886 date: 2018/08/06 15:02:51; author: gilles; state: Exp; lines: +28 -11 -Added mygetppid because USR1 no available on Windows. +Added mygetppid because getppid is not available on Windows. Removed --nosubscribed since it is not coded and useless anyway. ---------------------------- revision 1.885 diff --git a/FAQ.d/FAQ.Admin_Authentication.txt b/FAQ.d/FAQ.Admin_Authentication.txt index 1fee661..b23ec77 100644 --- a/FAQ.d/FAQ.Admin_Authentication.txt +++ b/FAQ.d/FAQ.Admin_Authentication.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Admin_Authentication.txt,v 1.6 2018/09/09 10:58:16 gilles Exp gilles $ +$Id: FAQ.Admin_Authentication.txt,v 1.9 2019/06/26 22:16:43 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -11,30 +11,43 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt ======================================================================= It can be useful to authenticate without knowing each user password. + Using an admin account to authenticate is not a standard feature supported by all imap servers. Sometimes it is implemented via --authuser1 -sometimes not. It depends on the server software and its configuration. -You have to figure out what is the imap software server, -find out if it supports an admin account and how, -if an admin account is actually configured and -finally check an authentication via this admin account for -a standard user. +sometimes not. It depends on the imap software server and its configuration. -Known imap server software supporting imap admin authentication: +You have to follow this road: + * Determine what is the imap software server + * Find out wether it supports an admin account or not + * If no, leave this reading, take an handkerchief and cry + * If yes, search how the imap software server supports + an admin account (see below) + * Figure out wether an admin account is actually configured or not, + * If not then configure it, + * Finally, check an imapsync authentication via this admin account for + a standard user, option --justlogin is nice for doing this. + * Use it for all users. + * Next, read their email messages, enter their personnal life etc. + I'm kidding. - * Exchange 2003/2007/2010/2013. See the file FAQ.Exchange.txt - * Office365. See the file FAQ.Exchange.txt - * Gmail. See the file FAQ.XOAUTH2.txt - * Dovecot. See the file FAQ.Dovecot.txt - * Zimbra. See the file FAQ.Zimbra.txt - * UW-imap. See the file FAQ.General.txt +Good luck! + +Known imap software servers supporting imap admin authentication: + + * Exchange 2003/2007/2010/2013/2016. See the file FAQ.Exchange.txt + * Office365. See the file FAQ.Exchange.txt + * Gmail. See the file FAQ.XOAUTH2.txt + * Dovecot. See the file FAQ.Dovecot.txt + * Zimbra. See the file FAQ.Zimbra.txt + * Kerio See the file FAQ.Kerio.txt + * UW-imap. See the file FAQ.General.txt * Cyrus-imap. See the file FAQ.General.txt * Sun Java Enterprise System/SunOne/iPlanet. See the file FAQ.General.txt -Known imap server software NOT supporting imap admin authentication: +Known imap software servers NOT supporting imap admin authentication: - * Domino Lotus Notes. - * SmartCloud Notes. + * Domino Lotus Notes. See FAQ.Handkerchief.txt + * SmartCloud Notes. Same joke. ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Archiving.txt b/FAQ.d/FAQ.Archiving.txt index d551a04..cd56d16 100644 --- a/FAQ.d/FAQ.Archiving.txt +++ b/FAQ.d/FAQ.Archiving.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Archiving.txt,v 1.13 2018/12/09 21:26:25 gilles Exp gilles $ +$Id: FAQ.Archiving.txt,v 1.14 2019/03/22 11:50:36 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -24,6 +24,8 @@ Q. Can imapsync be used to maintain a local offline copy of a ideally without requiring a second IMAP host? + +https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt ======================================================================= Q. Can I archive different accounts on the same destination account, each account on a separate folder? @@ -37,15 +39,16 @@ R. Yes. Use --subfolder2 Example: - imapsync ... --user1 foo --subfolder2 foo + imapsync ... --user1 Foo --subfolder2 FooArch - imapsync ... --user1 bar --subfolder2 bar + imapsync ... --user1 Bar --subfolder2 BarArch -In case you need a strict sync, add --delete2 --delete2foldersonly "foo" -(or "bar"), il will delete on account2 what is not on account1 but only -in the right place, ie, the subfolder "foo". +In case you need a strict sync, add --delete2 --delete2foldersonly "FooArch" +(or "BarArch" for the second example); it will delete on account2 +what is not on account1 but only in the right place, ie, the subfolder "foo". +Example: - imapsync ... --user1 foo --subfolder2 foo --delete2 --delete2foldersonly "foo" + imapsync ... --user1 Foo --subfolder2 FooArch --delete2 --delete2foldersonly "FooArch" ======================================================================= Q. How to move emails from one IMAP folder to another either on the diff --git a/FAQ.d/FAQ.Contacts_Calendars.txt b/FAQ.d/FAQ.Contacts_Calendars.txt index 1709919..f214c34 100644 --- a/FAQ.d/FAQ.Contacts_Calendars.txt +++ b/FAQ.d/FAQ.Contacts_Calendars.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Contacts_Calendars.txt,v 1.11 2018/09/26 22:25:32 gilles Exp gilles $ +$Id: FAQ.Contacts_Calendars.txt,v 1.12 2019/04/30 13:04:18 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -26,21 +26,25 @@ R. No, Imapsync can't migrate Contacts and Calendars. events via IMAP. In other words, messages synced by imapsync from Calendars or Contacts folders are not used by email servers to set or get the contacts or calendars. - No way via IMAP, no way via imapsync. - So it's a good idea to avoid syncing contacts and calendars. - But see the question below + No way via IMAP, no way via imapsync. + + So it's a good idea to avoid syncing contacts and calendars when + using imapsync. See below + "How can I avoid copying Calendar or Contacts folders?" + + But see also the question at the end of this document "How can I copy or synchronize Calendars or Contacts?". ======================================================================= Q. Can I copy or sync Chat messages with imapsync? -R. No, Imapsync can't migrate Chat messages (those in GSuite for example) +R. No, Imapsync can't migrate Chat messages (those in GSuite for example). The explanation is the same as with Calendars or Contacts. ======================================================================= Q. How can I avoid copying Calendar or Contacts folders? -R1. You can avoid synchronizing Calendar or Contacts folders with +R. You can avoid synchronizing Calendar or Contacts folders with the --exclude option. First you have to search what is their exact name. The folders listing printed by imapsync at the beginning of a run will surely help to find their names. @@ -48,6 +52,25 @@ R1. You can avoid synchronizing Calendar or Contacts folders with imapsync ... --exclude "Calendar" --exclude "Contacts" + Be aware that this syntax will exclude all folders that have + Calendar or Contacts in their names, as a substring, + like "MyCalendarZ" or "TheContactsImDoneWith". + Also, case matters: "calendar" does not match "Calendar". + + To stritly avoid only "Calendar" or "Contacts" use + the following + + On windows: + + imapsync ... --exclude "^Calendar$" --exclude "^Contacts$" + + On Unix: + + imapsync ... --exclude '^Calendar$' --exclude '^Contacts$' + or + imapsync ... --exclude "^Calendar\$" --exclude "^Contacts\$" + + ======================================================================= Q. How can I copy or synchronize Calendars or Contacts? diff --git a/FAQ.d/FAQ.Dates.txt b/FAQ.d/FAQ.Dates.txt index 49d01b4..1059d3d 100644 --- a/FAQ.d/FAQ.Dates.txt +++ b/FAQ.d/FAQ.Dates.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Dates.txt,v 1.9 2018/11/04 09:27:17 gilles Exp gilles $ +$Id: FAQ.Dates.txt,v 1.10 2019/05/27 14:26:39 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -34,15 +34,17 @@ but not with: - Zimbra - Gmail -A first thing to keep in mind. -Imapsync does not touch any byte of messages unless told to -do so by special options like --regexmess or --pipemess. -Messages on both parts should be exactly identical because -imapsync does a verbatim sync of all messages. +First, a thing to keep in mind. + +Imapsync does not touch any byte of messages unless told to do so by +special options like --regexmess or --pipemess. Messages on both +parts should be exactly identical because imapsync does a verbatim +sync of all messages, by default. Now I explain the whole picture about dates of messages. There are several different dates attached to any message. + First, there is the "Date:" header, which belongs to the message itself. You can see this "Date:" header line by asking "View->Message Source" in most email client software (Ctrl-u in Thunderbird). @@ -52,37 +54,40 @@ MUA can be Outlook, Thunderbird, a webmail or Mutt etc. The "Date:" header is usually the date the message was written or sent the first time, at the writer side. This "Date:" header line is never changed by any transfer or copy. -If an email reader uses the "Date:" header for displaying the date of a -message then no problem should arise. -Another date is the internal date. In IMAP, the internal date is handled -by the imap server and normally it corresponds to the arrival date in the mailbox. -This date is not part of the email message itself, it's a metadata kept -by the imap server. -The IMAP protocol allows the internal date to be set by a email client +If an email reader uses the "Date:" header for displaying the date of +a message then no problem should arise. + +Another date is the "internal date". In IMAP, the internal date is +handled by the imap server and normally it corresponds to the arrival +date in the mailbox. This date is not part of the email message +itself, it is a metadata kept by the imap server. + +The IMAP protocol allows the internal date to be set by an email client when a message is delivered by the imap APPEND command. Imapsync synchronizes these internal dates, by default, so internal dates on host2 should then be the same as the internal dates on host1. -If an email reader uses the internal date for displaying the date of a -message then the sync date problem only occurs when the host2 server software -ignores the internal date given by imapsync during the APPEND imap -command. It happens with some imap servers. -There are also the "Received:" header lines. Each time a message travels -a SMTP server, this one adds a "Received:" header line. Sometimes -some email clients use the last "Received:" header date as the date of the -message. And some IMAP servers software add a "Received:" line after -and imap transfer. If those both conditions are met then the date -displayed become the transfer date even if imapsync +If an email reader uses the internal date for displaying the date of a +message then the sync date problem only occurs when the host2 imap +server ignores the internal date given by imapsync during the APPEND +imap command. It happens with some imap servers. Bad luck. + +There are also the "Received:" header lines. Each time a message +travels a SMTP server, this one adds a "Received:" header +line. Sometimes some email clients use the last "Received:" header +date as the date of the message. And some IMAP servers software add a +"Received:" line after and imap transfer. If those both conditions are +met then the date displayed become the transfer date even if imapsync has done its best to keep all the dates synchronized. Bad luck. Solutions: a) Use another email client or configure it in order that it sorts messages -by sent dates, the "Date:" header line. + by sent dates, the "Date:" header line. b) Use a imap server that respects the imap RFC and accepts -the internal date set by imapsync. + the internal date set by imapsync. c) Try to understand why the email client software shows another date than the "Date:" header line. diff --git a/FAQ.d/FAQ.David_Tobit.txt b/FAQ.d/FAQ.David_Tobit.txt new file mode 100644 index 0000000..f1fbc70 --- /dev/null +++ b/FAQ.d/FAQ.David_Tobit.txt @@ -0,0 +1,54 @@ +#!/bin/cat +$Id: FAQ.David_Tobit.txt,v 1.4 2019/03/28 17:58:53 gilles Exp gilles $ + +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.David_Tobit.txt + + +======================================================================= + Imapsync tips for David Tobit imap server +======================================================================= + + + +======================================================================= +Q. Migrating to David Tobit V12 + (Banner: "* OK IMAP4rev1 David.fx Mail Access Server MA-12.00a (0271)") + + imapsync ... --sep2 / --create_folder_old + +Thanks to Ludwig Behm + +======================================================================= +Q. Migrating from David Tobit V10 (DvISE Mail Access Server MA-...) + +R. Use the following options: + + imapsync ... --prefix1 "" --sep1 / --idatefromheader ^ + --nofoldersizes --useuid --nocheckmessageexists + +======================================================================= +Q. Migrating from David Tobit V8 + (Banner: "* OK IMAP4rev1 DvISE Mail Access Server MA-8.10a (0126)") + +First try above V10 solution since improvments have been made +to support Tobit. + +R. Use the following options : + + imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed + +======================================================================= +Q. Migrating from Tobit David Server 6 + (Banner: "* OK IMAP4rev1 DvISE Mail Access Server MA-6.60a (0118)") + +First try above V10 solution since improvments have been made +to support Tobit. + +R. Look at the discussion: +http://www.linux-france.org/prj/imapsync_list/msg00583.html +http://www.linux-france.org/prj/imapsync_list/threads.html#00583 +patch saved in ./patches/imapsync-1.337_tobit_V6.patch + +======================================================================= \ No newline at end of file diff --git a/FAQ.d/FAQ.Duplicates.txt b/FAQ.d/FAQ.Duplicates.txt index 5d69b7d..5a1f911 100644 --- a/FAQ.d/FAQ.Duplicates.txt +++ b/FAQ.d/FAQ.Duplicates.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Duplicates.txt,v 1.18 2019/02/14 17:22:10 gilles Exp gilles $ +$Id: FAQ.Duplicates.txt,v 1.20 2019/05/17 10:10:30 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -13,8 +13,9 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt Q. How can I know if imapsync will generate duplicates on a second run? R. To see if imapsync will generate duplicates on a second run, start - a second run with --dry option added. imapsync will then show if it - would mistakenly copy messages again, but without really copying them + a second run with --dry option added. With --dry, imapsync will + show whether it would mistakenly copy messages again, but without + really copying them: imapsync ... --dry @@ -35,7 +36,7 @@ Q: Multiple copies, duplicates, when I run imapsync twice ore more. R0. First, some explanations to understand the issue. Normally and by default, imapsync doesn't generate duplicates. -So if it does generate duplicates it means a problem occurs +So, if it does generate duplicates it means a problem occurs with message identification. It happens sometimes with IMAP servers changing the "Message-Id" header line or one or more of the "Received:" header lines in the header part of messages. @@ -66,7 +67,7 @@ don't go on, it means imapsync is still suffering to identify messages. If you end with many messages skipped then it's very -good and now you can safely resync the mailboxe +good and now you can safely resync the mailbox and get rid of the dupplicates messages on host2 with: imapsync ... --useheader "Message-Id" --delete2duplicates @@ -80,7 +81,7 @@ to identify and compare messages in folders. Instead of some headers, --useuid tell imapsync to use the imap UIDs given by imap servers on both sides. To avoid duplicates on next runs, imapsync uses a local cache -where it keeps UIDs already transfered. +where it keeps UIDs already transferred. imapsync ... --useuid @@ -95,11 +96,12 @@ A solution? Two solutions. The easiest is --delete2 if you are permitted to use it. Option --delete2 removes messages on host2 that are not on host1. So, with --delete2 you go for resyncing all -messages again but all previously transferred messages are deleted, -but also messages previously there without imapsync. -So --useuid --delete2 is easy to remove duplicates but not for -all contexts. The host2 account must be considered as a strict -replication of the host1 account, ie, not active. +messages again. All previously transferred messages are deleted, +but also messages previously there without imapsync. +So --useuid --delete2 is an easy way to remove duplicates but it +is not suitable in all contexts. The good context is that the host2 +account must be considered as a strict replication of the host1 +account, ie, host2 not active yet. A second solution, better if R3 works (see R3 below), is to build the cache before using --useuid @@ -191,8 +193,11 @@ R2. With option --useuid imapsync doesn't use headers to identify ======================================================================= Q. How can I remove duplicates in a lone account? -R. Just run imapsync on the same account with option --delete2duplicates, - ie, with host1 == host2, user1 == user2, password1 == password2 +R. In order to remove duplicates in a lone account, just run imapsync + on the same account as source and destination, plus the + option --delete2duplicates, ie, with + host1 == host2, user1 == user2, password1 == password2 + ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Emptying.txt b/FAQ.d/FAQ.Emptying.txt index 9ae1e85..3cf9c26 100644 --- a/FAQ.d/FAQ.Emptying.txt +++ b/FAQ.d/FAQ.Emptying.txt @@ -1,9 +1,10 @@ #!/bin/cat -$Id: FAQ.Emptying.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $ +$Id: FAQ.Emptying.txt,v 1.8 2019/04/30 12:52:10 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ -https://imapsync.lamiral.info/FAQ.d/ + +https://imapsync.lamiral.info/FAQ.d/FAQ.Emptying.txt ======================================================================= Imapsync tip to empty an account. @@ -12,27 +13,28 @@ https://imapsync.lamiral.info/FAQ.d/ ======================================================================= Q. How to delete all emails of all folders of an account with imapsync? -R. Use the same account on both sides with option --delete +R. Use the same account on both sides with option --delete1 --noexpungeaftereach -Option --noexpungeaftereach is just to speed up the deletion. +Option --noexpungeaftereach is just to speed up the deletions. -Example: +Example (real example): ./imapsync \ - --host1 test.lamiral.info --user1 test --password1 secret \ - --host2 test.lamiral.info --user2 test --password2 secret \ - --delete --noexpungeaftereach + --host1 test.lamiral.info --user1 empty --password1 secret \ + --host2 test.lamiral.info --user2 empty --password2 secret \ + --delete1 --noexpungeaftereach ======================================================================= Q. How to delete all folders of an account with imapsync? -R. Use an already empty account as source and sync this account - to the account you want to clear, with --delete2folders +R. Use the same account on both sides with the options + --delete1 --noexpungeaftereach --delete1emptyfolders + + ./imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders -Example: +It won't delete the folder INBOX since INBOX is mandatory in imap. - ./imapsync ... --delete2folders --foldersizes ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Exchange.txt b/FAQ.d/FAQ.Exchange.txt index aa78fe5..cc3148c 100644 --- a/FAQ.d/FAQ.Exchange.txt +++ b/FAQ.d/FAQ.Exchange.txt @@ -1,5 +1,5 @@ -$Id: FAQ.Exchange.txt,v 1.49 2018/12/03 23:09:28 gilles Exp gilles $ +$Id: FAQ.Exchange.txt,v 1.50 2019/04/02 09:29:38 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -455,6 +455,15 @@ encountered so you might also see "BYE Connection closed" errors from Exchange, which means Exchange leaves the session and say goodbye, come back later. Rerun a sync then. +If on Linux, there is a good Python script in the tarball that can +fix several things that Exchange or O365 have issues with. + +Use it like this: + + ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py + +It often does some miracles on messages. + ======================================================================= Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does not seem to be well synced. What can I do? diff --git a/FAQ.d/FAQ.Flags.txt b/FAQ.d/FAQ.Flags.txt index e9760ff..9f56bf3 100644 --- a/FAQ.d/FAQ.Flags.txt +++ b/FAQ.d/FAQ.Flags.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Flags.txt,v 1.20 2019/02/14 16:01:41 gilles Exp gilles $ +$Id: FAQ.Flags.txt,v 1.21 2019/03/08 19:04:32 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -149,22 +149,22 @@ Add the \Seen flag to all messages like this: On Winwows: - imapsync.exe ... --regexflag "s,^((?!\\Seen)).*$,$1 \\Seen," + imapsync.exe ... --regexflag "s,\\Seen,," --regexflag "s,,\\Seen ," + +On Unix: + + imapsync ... --regexflag 's,\\Seen,,' --regexflag 's,,\\Seen ,' + +R2. You can also filter with --search1 UNSEEN and use only one + of the regexes: On Unix: - imapsync ... --regexflag 's,^((?!\\Seen)).*$,$1 \\Seen,' - -R2. You can also filter with --search1 UNSEEN and use a simpler - regex: + imapsync ... --search1 UNSEEN --regexflag 's,,\\Seen ,' On Winwows: - imapsync.exe ... --search1 UNSEEN --regexflag "s/(.*)/$1 \\Seen/" - -On Unix: - - imapsync ... --search1 UNSEEN --regexflag 's/(.*)/$1 \\Seen/' + imapsync.exe ... --search1 UNSEEN --regexflag "s,,\\Seen ," R3. Fix it on the server Exchange: Google translate: diff --git a/FAQ.d/FAQ.Folders_Mapping.txt b/FAQ.d/FAQ.Folders_Mapping.txt index 96fc0c7..d33b6ba 100644 --- a/FAQ.d/FAQ.Folders_Mapping.txt +++ b/FAQ.d/FAQ.Folders_Mapping.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Folders_Mapping.txt,v 1.17 2018/11/20 10:26:18 gilles Exp gilles $ +$Id: FAQ.Folders_Mapping.txt,v 1.19 2019/03/04 21:21:18 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -39,16 +39,17 @@ Things to know and understand before playing with --regextrans2 *) --regextrans2 is used to transform folders names. *) --regextrans2 applies after the default - inversion prefix1 <-> prefix2 and sep1 <-> sep2 - So when elaborating the regex you should focus on + inversion prefix1 <-> prefix2 and sep1 <-> sep2. + So, when elaborating the regex you should focus on the right part of the default mapping printed by imapsync, the part showing the host2 folder name. - The section to look at is within the folder loop: + The section to look at is either the folder size section + or within the folder loop located "Here" between the lines ++++ Looping on each folder Here ++++ End looping on each folder - -*) Several --regextrans2 is possible, they will be applied in the order + +*) Several --regextrans2 are possible, they will be applied in the order of the command line, each one on the result of the previous one. *) --regextrans2 uses Perl regex mechanism so it may be hard to master @@ -61,18 +62,19 @@ Here On Linux/Unix use single quotes ' around the regex string, it is easier to get what we want with single quotes since the shell - won't change the inner string. Like --regextrans2 'myregex' + won't change the inner string. Use --regextrans2 'myregex' *) Good method to elaborate any --regextrans2 string First, elaborate the --regextrans2 string with --dry --justfolders options. - imapsync ... --dry --justfolders + imapsync ... --dry --justfolders With --dry imapsync shows the transformations it will do without really doing them, --dry is the "do nothing" mode. With --justfolders imapsync will work only with folders, - messages won't be taken into account, so it will be fast. + messages won't be taken into account, so it will be fast + and focused on the folders names. When the output shows what you expect imapsync to do with folders names, you can remove the --dry option. Keep the --justfolders @@ -106,50 +108,13 @@ Examples: 2b) To sync a complete account in a subfolder called FOO -Since imapsync release 1.641 simply use +With imapsync release 1.641 and former, simply use imapsync ... --subfolder2 FOO -Next examples are subfolder solutions for any release. +Examples to this subfolder problem for any release +can be seen below, in another Q/R section. - a) Separator is dot character "." and "INBOX" prefixes every folder - -On Linux/Unix: - - --regextrans2 's,^INBOX(.*),INBOX.FOO$1,' - -On Windows: - - --regextrans2 "s,^INBOX(.*),INBOX.FOO$1," - - or: - - b) Separator is the slash character "/" and there is no prefix - -On Linux/Unix: - - --regextrans2 's,(.*),FOO/$1,' - -On Windows: - - --regextrans2 "s,(.*),FOO/$1," - - or: - - c) Any separator, any prefix solution, FOO is the subfolder: - - It is a complicated line because every case is taken into account. - Type it in one line (or with the \ at the end of first line on Unix shells. - -On Linux/Unix: - - --regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \ - --regextrans2 's,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,' - -On Windows: - - --regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," ^ - --regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX," 3) to substitute all characters dot "." by underscores "_" --regextrans2 "s,\.,_,g" @@ -164,8 +129,7 @@ On Windows: --regextrans2 s,\^",_,g -3c) to substitute all characters *%. by underscores _ -You can increase the *%. list by any unwanted character. +3c) to substitute all characters * % . and # by underscores _ On Linux/Unix: @@ -175,11 +139,13 @@ On Windows: --regextrans2 "tr,*%.#,_," +You can increase the *%.# list by any unwanted character, +plus, the order doesn't count. See 3d) for & + 3d) It is a bad idea to substitute & characters since & is a character to encode non-ascii characters in IMAP folder names. - 4) to change folder names like this: [mail/Sent Items] -> [Sent] [mail/Test] -> [INBOX/Test] @@ -214,39 +180,26 @@ to folder INBOX only on host2: 4) Remove --justfolders -====================================================================== -Q. I have moved from Braunschweig to Graz, so I would like to have my - whole Braunschweig mail sorted into a sub-folder INBOX.Braunschweig - of my new mail account. - -R. -1) First try (safe mode): - -imapsync \ - ... - --regextrans2 "s/INBOX(.*)/INBOX.Braunschweig\$1/" \ - --dry --justfolders - -On Windows, in the previous example containing \$1 you have to -replace the two \$1 by $1 (remove the \ before $). - - -2) See if the output says everything you want imapsync to do, - --dry option is safe and does nothing real. - -3) Remove --dry - Check the imap folder tree on the target side - -4) Remove --justfolders - ======================================================================= Q. I would like to move emails from InBox to a sub-folder called, - say "2010-INBOX" based on the date (Like all emails received in the - Year 2010 should be moved to the folder called "2010-INBOX"). + say "2010-INBOX" based on the date. Like all emails received in the + year 2010 should be moved to the folder called "2010-INBOX". R. 2 ways : -a) Manually: +a) With imapsync: +----------------- + + imapsync ... \ + --search "SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010" + --f1f2 "INBOX=2010-INBOX" \ + --folder INBOX + +See also the FAQ FAQ.Messages_Selection.txt +https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt +to get more examples about how to select messages. + +b) Manually: ------------ 1) You create a folder INBOX.2010-INBOX @@ -257,13 +210,58 @@ a) Manually: 3) Cut/paste in INBOX.2010-INBOX -b) With imapsync: ------------------ - -imapsync ... \ ---search 'SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010' ---regextrans2 's/^INBOX$/INBOX.2010-INBOX/' \ ---folder INBOX +======================================================================= +Q. How to sync a complete account in a subfolder called FOO, + the hard way. + +R0. See the easy above using --subfolder2 + +R1. Several ways to do it. + + a) Separator is dot character "." and "INBOX" prefixes every folder + +On Linux/Unix: + + --regextrans2 's,^INBOX(.*),INBOX.FOO$1,' + +On Windows: + + --regextrans2 "s,^INBOX(.*),INBOX.FOO$1," + + or: + + b) Separator is the slash character "/" and there is no prefix + +On Linux/Unix: + + --regextrans2 's,(.*),FOO/$1,' + + or with doublequotes + + --regextrans2 "s,(.*),FOO/\$1," + +On Windows: + + --regextrans2 "s,(.*),FOO/$1," + + or: + + c) Any separator, any prefix solution, FOO is the subfolder: + + It is a complicated line because every case is taken into account. + Type it in one line (or with the \ at the end of first line on Unix shells. + +On Linux/Unix: + + --regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \ + --regextrans2 's,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,' + +On Windows: + + --regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," ^ + --regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX," + + ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Gmail.txt b/FAQ.d/FAQ.Gmail.txt index ff7d108..7ecc725 100644 --- a/FAQ.d/FAQ.Gmail.txt +++ b/FAQ.d/FAQ.Gmail.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Gmail.txt,v 1.51 2019/02/18 09:49:14 gilles Exp gilles $ +$Id: FAQ.Gmail.txt,v 1.58 2019/04/25 10:41:12 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -67,6 +67,14 @@ it has to be allowed in the Gmail configuration part: -> IMAP Access -> Enable IMAP + Also: + + -> Verify 2-step verification is OFF + https://myaccount.google.com/security?hl=en&pli=1&nlr=1#signin + -> Turn ON "Access for less secure apps" + https://www.google.com/settings/security/lesssecureapps + https://support.google.com/accounts/answer/6010255?hl=en + ======================================================================= Q. Even after several syncs the number of messages differs largely between imapsync and the Gmail web interface. @@ -89,10 +97,10 @@ R. Basically it takes X days to transfer X GB per account. From the previous link: * it's 2X days to upload X GB to Gmail, it's why I suggest to add - --maxbytespersecond 10000 + --maxbytespersecond 10000 --maxbytesafter 1_000_000_000 for uploading messages to Gmail * it's X/2 days to download X BG from Gmail, it's why I suggest to add - --maxbytespersecond 20000 + --maxbytespersecond 20000 --maxbytesafter 1_000_000_000 for downloading messages from Gmail That's theoretical values that always work in practice. Try @@ -192,9 +200,9 @@ imapsync --host1 mail.oldhost.com \ --password1 password \ --user2 my_email@gmail.com \ --password2 password \ - --gmail2 --maxsize 35_651_584 + --gmail2 -Or, replacing what does --gmail2 (values are from imapsync 1.882): +Or, replacing what does --gmail2 (values are from imapsync 1.921): imapsync --host1 mail.oldhost.com \ --user1 my_email@oldhost.com \ @@ -209,9 +217,27 @@ imapsync --host1 mail.oldhost.com \ --automap \ --expunge1 \ --addheader \ - --exclude "\[Gmail\]$" \ - --regextrans2 "s/[ ]+/_/g" \ - --regextrans2 "s/['\^\"\\\\]/_/g" + --exclude "\[Gmail\]$" \ + --regextrans2 "s/['\^\"\\\\]/_/g" \ + --regextrans2 "s/[ ]+/_/g" + + +Starting from imapsync 1.926 the last --regextrans2 replacing +all blanks, which was: + + --regextrans2 "s/[ ]+/_/g" + +became + +on Winwows, like: + + imapsync.exe ... --regexflag "s,\\Seen,," --regexflag "s,,\\Seen ," + +on Linux, like: + + imapsync ... --regexflag 's,\\Seen,,' --regexflag 's,,\\Seen ,' + + With --gmail2 you can override the parameters that are activated by them, for example: @@ -438,9 +464,10 @@ R. No. using the IMAP protocol. ======================================================================= -Q. Can I safely use --useuid for Gmail transfers? +Q. Can I safely use --useuid or --usecache for Gmail transfers? -R. Yes, but I suggest to not use --useuid for Gmail transfers. +R. Yes, but I suggest to not use --useuid nor --usecache + for Gmail transfers. Using UIDs is useless with Gmail in the case of global duplicates (duplicates across different folders). Gmail always accept a global @@ -546,7 +573,6 @@ xxx sub-folders show up nested within xxx/xxx. Q. How to migrate email from gmail to google apps? R. Take a look at: -http://www.linux-france.org/prj/imapsync_list/msg00639.html http://biasecurities.com/2009/02/migrate-email-from-gmail-to-google-apps/ http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration/ diff --git a/FAQ.d/FAQ.Kerio.txt b/FAQ.d/FAQ.Kerio.txt new file mode 100644 index 0000000..be738c3 --- /dev/null +++ b/FAQ.d/FAQ.Kerio.txt @@ -0,0 +1,40 @@ +#!/bin/cat +$Id: FAQ.Kerio.txt,v 1.1 2019/06/26 22:14:50 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Kerio.txt + +======================================================================= + Imapsync tips for Kerio accounts. +======================================================================= + + + + +======================================================================= +Q. How to migrate from Kerio Connect to XXX + +R. No special options required. +See also: +http://www.linux-france.org/prj/imapsync_list/msg01756.html +http://web.archive.org/web/20140224041102/https://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-1/ +http://web.archive.org/web/20121125074756/http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-2/ + +======================================================================= +Q. How to use Kerio master authentication with imapsync? + +R. Example: + +Kerio at host1 + + imapsync ... --user1 USERNAME --password1 MASTERPASSWORD --authmech1 X-MASTERAUTH + +Kerio at host2 + + imapsync ... --user2 USERNAME --password2 MASTERPASSWORD --authmech2 X-MASTERAUTH + +See also: +https://github.com/imapsync/imapsync/pull/53 + +======================================================================= \ No newline at end of file diff --git a/FAQ.d/FAQ.Massive.txt b/FAQ.d/FAQ.Massive.txt index 40a2d2b..a456543 100644 --- a/FAQ.d/FAQ.Massive.txt +++ b/FAQ.d/FAQ.Massive.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Massive.txt,v 1.24 2018/12/30 21:28:03 gilles Exp gilles $ +$Id: FAQ.Massive.txt,v 1.29 2019/06/18 16:38:06 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -29,18 +29,46 @@ Q. I run multiple imapsync applications at the same time then get a IMAP account in parallel? ======================================================================= -Q. How long will take the whole migration? +Q. How long will take the whole migration? -R. Consider the mean imapsync transfer rate to be around 340 Kbytes/s, - ie, 2.8 Mbps, no matter the local link bandwidth. - It's a mean, measured upon various different syncs, coming from - the online interface /X. - - With 1 TB to transfer and one sync at a time it will end in 35 days. - At 10 transfers at a time, 1 TB will take 3.5 days. - Double the time because best scenario never happens. - Triple because, well, real world is like that. +R1. First you have to consider several periods. There is the global + period, from when the migration process is decided to the final end + where all mailboxes are migrated. This global period can be divided + into three smaller periods. + + The first period is the analyse period: you play with the tools + available, you estimate the volume to be transferred, the number of + accounts, you measure how long it takes for one account under + your context. + + The second period is the presync period. The users are still using + the old accounts but nothing prevents you from starting to sync + the old accounts, as they are, to the new accounts. + With tons of gigabytes to transfer, this period may be the longest one. + There is nothing more to launch the presyncs and watch until + the round is finished. + + The last period is the final sync period where only the last + changes of the mailboxes need to be synced in order to switch + the users to their new mailboxes. + +R2. To estimate the presync period, consider the mean imapsync transfer + rate to be around 340 Kbytes/s, ie, 2.8 Mbps, no matter the local + link bandwidth. + It's a mean, measured upon various different syncs, coming from + the online interface /X. + At 340 Kbytes/s, + 1 TB to transfer and one sync at a time will end in 35 days (1024^3/340/3600/24). + At 10 transfers at a time, 1 TB will take 3.5 days. + At 100 transfers at a time, 1 TB will take 8 hours. + Double the time because the best scenario never happens. + Triple the time because, well, real world is like that. + +R3. To estimate the last period, just rerun a complete presync, + ie, resync all the mailboxes, the final sync should take + the same amount of time. + ======================================================================= Q. I need to migrate hundred accounts, how can I do? @@ -119,43 +147,43 @@ of sync_loop_windows.bat Q. I have to migrate 500k users using 400 TB of disk space. How do I proceed? How about speed? -R. Solution to this issue is two words: parallelism and measurements. +R. A good solution to this issue is two words: parallelism and measurements. Since all mailboxes are functionnaly independent, they can be processed -independently, here comes parallelism, lunching several imapsync +independently, here comes the parallelism, ie, lunching several imapsync processes in parallel. -Meanwhile, mailboxes usually belong to the same server and syncs +Meanwhile, mailboxes usually belong to the same server and the syncs share the same imapsync host via the same bandwidth, here come some limitations and bottlenecks. -How many syncs can we run in parallel? here comes measurements. +How many syncs can we run in parallel in your context? +Here comes some measurements. 1) Measure the total transfer rate by adding each one printed in each run. Since adding this way is not so easy, just look at the overall network rate of the imapsync host. On Linux and FreeBSD, nload is a good candidate to measure this overall - network rate, every 6 seconds (-t 6000), on eth0 or em0 interface, - values in Kbytes (-u K): + network rate. For example, to measure the rate every 6 seconds (-t 6000), + on eth0 or em0 interface, with values in Kbytes (-u K), use: nload -t 6000 eth0 -u K # Linux nload -t 6000 em0 -u K # FreeBSD - During nload, press the h to see the display commands available, - every single feature is useful! Press h again and try each one. - On Linux only, another very good network tool is dstat: dstat -n -N eth0 6 # Linux only (in 2018) - A excellent tool for this purpose is iftop, the following - command will monitor imap and imaps connexions + Another excellent tool to measure the network trafic is iftop. + The following command will monitor imap and imaps connexions on interfce eth0, only them, and sum them up: iftop -i eth0 -f 'port imap or port imaps' -B # Linux iftop -i em0 -f 'port imap or port imaps' -B # FreeBSD + During iftop, press the h to see the display commands available, + every single feature is useful! Press h again and try each one. On Windows 8.1 Windows 10 Windows 2012 R2 Windows 2016, @@ -232,9 +260,11 @@ Possible bottlenecks: - RAM memory. - On all sides, monitor that your systems don't swap on disk, - because swapping memory on disks decreases performance by - a factor of 20, at least. + On all sides, monitor that your systems don't swap its + running processes on disk, because swapping running processes + on disks decreases performance by a factor of 20, at least. + It's not because the swap memory is used that your + system swaps processes on disk. - CPU. 100% CPU during a whole transfer means the system is busy. @@ -245,8 +275,9 @@ Possible bottlenecks: Other possible bottlenecks: - Number of hosts available to run imapsync processes. - Imapsync itself. -- Errors management. +- Management of errors. - MX domains, DNS. +- Incompetence. - Money. - Time. - Bad luck. @@ -263,7 +294,7 @@ Q. Any performance issue? You have to try and check the transfer rates, sum them up to have a uniq numeric criteria. There is always a limit, depending on remote imap servers -and the one running imapsync; +and the one running imapsync. CPU, memory, Inputs/Outputs are the classical bottlenecks, the worst bottleneck is the winner that sets the limit. diff --git a/FAQ.d/FAQ.Messages_Too_Big.txt b/FAQ.d/FAQ.Messages_Too_Big.txt new file mode 100644 index 0000000..58f9e47 --- /dev/null +++ b/FAQ.d/FAQ.Messages_Too_Big.txt @@ -0,0 +1,51 @@ +#!/bin/cat +$Id: FAQ.Messages_Too_Big.txt,v 1.3 2019/06/18 16:37:39 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt + +======================================================================= + Dealing with too big messages with Imapsync. +======================================================================= + +Questions answered in this FAQ are: + +Q. What can I do to transfer messages bigger than what allows the + imap destination server? + +R1. A solution is to truncate the message to the maximum size allowed + before copying it to host2. That's a little dirty but it's ok + if you prefer having a truncated message than no message + at all. The attachments after the truncation won't be available + but everything before should be ok. It may also depends + on the software tool reading the email message. + +For example, +Gmail currently limits messages to 35651584 bytes (May 2019), + +Starting with imapsync release 1.938 option --truncmess xxxx truncates +messages bigger than the given size xxxx. + + imapsync --truncmess 35651584 --appendlimit 1000000000000 + +Prior to imapsync release 1.938, instead of --truncmess: + +On Linux, use: + + imapsync ... --pipemess 'perl -0ne "print substr \$_,0,35651584" ' + +On windows, use: + + I'm not sure of (not tested yet): + + imapsync ... --pipemess "perl -0ne 'print substr $_,0,35651584' " + + +R2. A not so dirty solution would be to transform the message and + include links to the big attachments, instead of the attachments. + Well, not done yet and too much complicted compared to the simple + solution which is to allow big messages on the destination account. + +======================================================================= +======================================================================= \ No newline at end of file diff --git a/FAQ.d/FAQ.Migration_Plan.txt b/FAQ.d/FAQ.Migration_Plan.txt index aa82985..9c5204f 100644 --- a/FAQ.d/FAQ.Migration_Plan.txt +++ b/FAQ.d/FAQ.Migration_Plan.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Migration_Plan.txt,v 1.5 2019/01/11 13:13:55 gilles Exp gilles $ +$Id: FAQ.Migration_Plan.txt,v 1.6 2019/04/10 12:05:31 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -85,7 +85,7 @@ Classical scenario, credentials triplets are different on both sides * In case there are still messages arriving at the old imap server host1, you can perform more postsyncs, ie, syncs every day with the options: - --maxage 1 --delete1 --folder INBOX. + --maxage 1 --delete1 --folder INBOX ===================================================================== diff --git a/FAQ.d/FAQ.Passwords_on_Unix.txt b/FAQ.d/FAQ.Passwords_on_Unix.txt index 28396ad..7106089 100644 --- a/FAQ.d/FAQ.Passwords_on_Unix.txt +++ b/FAQ.d/FAQ.Passwords_on_Unix.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Passwords_on_Unix.txt,v 1.3 2019/01/10 13:27:28 gilles Exp gilles $ +$Id: FAQ.Passwords_on_Unix.txt,v 1.6 2019/04/02 17:19:21 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -12,19 +12,29 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt ======================================================================= -Q. On Unix, some passwords contain some *(),;& characters. Login fails. +Q. On Unix, some passwords contain some weird *(),;&~ characters. + Login fails. R1. Enclose the password within single-quotes in the imapsync command line: - imapsync ... --password1 'passw*(),;&rd' + imapsync ... --password1 'passw*(),;&rd~' R2. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module, try also using double-quotes within single-quotes. It will enclose the password within double-quotes in the imap LOGIN command: - imapsync ... --password1 '"passw*(),;&rd"' + imapsync ... --password1 '"passw*(),;&rd~"' +Old Mail::IMAPClient 3.37 is a bad horse having this issue solved +by this double-quotes trick. +fixed in 3.38 https://metacpan.org/changes/distribution/Mail-IMAPClient +version 3.38: Tue Feb 9 02:48:21 UTC 2016 + - rt.cpan.org#110273: failure to quote password values + (regression introduced in 3.36 via fix for rt.cpan.org#100601) + +R3. Change the password to keep only f...ing normal characters. + Make it long and random if strong security is your concern ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Release_Checklist.txt b/FAQ.d/FAQ.Release_Checklist.txt index 6528ecc..e33cdc5 100644 --- a/FAQ.d/FAQ.Release_Checklist.txt +++ b/FAQ.d/FAQ.Release_Checklist.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Release_Checklist.txt,v 1.10 2019/02/14 15:52:44 gilles Exp gilles $ +$Id: FAQ.Release_Checklist.txt,v 1.11 2019/05/27 14:26:57 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -24,6 +24,10 @@ Checklist before release a new release: - Review the general FAQ.d/FAQ.General.txt +- /X verify direct cgi +- /X verify under noscript with firefox +- /X verify under private with chrome + - make dist - make publish - verify VERSION is uploaded diff --git a/FAQ.d/FAQ.SSL_errors.txt b/FAQ.d/FAQ.SSL_errors.txt index 272941e..33739dd 100644 --- a/FAQ.d/FAQ.SSL_errors.txt +++ b/FAQ.d/FAQ.SSL_errors.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.SSL_errors.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $ +$Id: FAQ.SSL_errors.txt,v 1.7 2019/04/05 20:04:38 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -67,14 +67,60 @@ R5. Set up a ssltunnel proxy to the host. Q. SSL connect attempt failed SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure -R. Use: +R1. Use: imapsync ... --sslargs1 'SSL_cipher_list=DEFAULT' or imapsync ... --sslargs2 'SSL_cipher_list=DEFAULT' depending on where the error occurs, host1 or host2 or both. - + +R2. If it doesn't work, I let you try other things, + I quote the ยง "SSL_version" section of + https://metacpan.org/pod/IO::Socket::SSL (Module version: 2.066) + + imapsync ... --sslargs1 SSL_version=SSLv2 + imapsync ... --sslargs1 SSL_version=TLSv1_2 + + SSLv2 and TLSv12 are just examples depending on your context + (--ssl1 or --tls1, and also the imap server encryption scheme) + +Feedback on what worked for you (and possibly hy) is welcome! + +https://metacpan.org/pod/IO::Socket::SSL +... +SSL_version + +Sets the version of the SSL protocol used to transmit data. +'SSLv23' uses a handshake compatible with SSL2.0, SSL3.0 and TLS1.x, +while 'SSLv2', 'SSLv3', 'TLSv1', 'TLSv1_1', 'TLSv1_2', or 'TLSv1_3' +restrict handshake and protocol to the specified version. +All values are case-insensitive. Instead of 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' +one can also use 'TLSv11', 'TLSv12', and 'TLSv13'. + +Support for 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' +requires recent versions of Net::SSLeay and openssl. + +Independent from the handshake format you can limit to set of +accepted SSL versions by adding !version separated by ':'. +The default SSL_version is 'SSLv23:!SSLv3:!SSLv2' which means, +that the handshake format is compatible to SSL2.0 and higher, +but that the successful handshake is limited to TLS1.0 and higher, +that is no SSL2.0 or SSL3.0 because both of these versions have +serious security issues and should not be used anymore. + +You can also use !TLSv1_1 and !TLSv1_2 to +disable TLS versions 1.1 and 1.2 while still allowing TLS version 1.0. + +Setting the version instead to 'TLSv1' might break interaction +with older clients, which need and SSL2.0 compatible handshake. + +On the other side some clients just close the connection +when they receive a TLS version 1.1 request. +In this case setting the version +to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help. + + ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.SmarterMail.txt b/FAQ.d/FAQ.SmarterMail.txt index b2ddd05..3544282 100644 --- a/FAQ.d/FAQ.SmarterMail.txt +++ b/FAQ.d/FAQ.SmarterMail.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.SmarterMail.txt,v 1.12 2018/05/24 11:34:30 gilles Exp gilles $ +$Id: FAQ.SmarterMail.txt,v 1.13 2019/06/20 10:11:21 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -17,21 +17,13 @@ Q. Synchronizing from SmarterMail to XXX On Unix: imapsync --host1 imap.d1.org --user1 joe --password1 secret1 \ --host2 imap.d2.org --user2 joe --password2 secret2 \ - --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch1 \ - --regextrans2 "s,Deleted Items,Trash," \ - --regextrans2 "s,Junk E-Mail,Junk," \ - --regextrans2 "s,Sent Items,Sent," + --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch1 On Windows: imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^ --host2 imap.d2.org --user2 joe --password2 secret2 ^ - --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch1 ^ - --regextrans2 "s,Deleted Items,Trash," ^ - --regextrans2 "s,Junk E-Mail,Junk," ^ - --regextrans2 "s,Sent Items,Sent," + --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch1 -Maybe add other --regextrans2 to change folder names, for this -see also http://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt ======================================================================= Q. Synchronizing from XXX to SmarterMail diff --git a/FAQ.d/FAQ.Various_Software_Servers.txt b/FAQ.d/FAQ.Various_Software_Servers.txt index 6ab9f66..6e7b622 100644 --- a/FAQ.d/FAQ.Various_Software_Servers.txt +++ b/FAQ.d/FAQ.Various_Software_Servers.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Various_Software_Servers.txt,v 1.10 2018/10/24 11:03:57 gilles Exp gilles $ +$Id: FAQ.Various_Software_Servers.txt,v 1.12 2019/06/26 22:17:03 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -13,12 +13,10 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Various_Server_Softwares.txt ======================================================================= -Q. From Zimbra to XXX +Q. From or to Zimbra -imapsync ... \ - --exclude "Conversation Action Settings" \ - --exclude "Quick Step Settings" \ - --exclude "News Feed" +R. Read the FAQ.Zimbra.txt + https://imapsync.lamiral.info/FAQ.d/FAQ.Zimbra.txt ======================================================================= Q. From or to HMailServer version 4.4.1. @@ -36,16 +34,6 @@ No specific option for HMailServer 5.3.3 since NAMESPACE is supported. Maybe --subscribe_all will help you to see all migrated folders. -======================================================================= -Q. Synchronizing from Kerio Connect to XXX - -R. No special options required. -See also: -http://www.linux-france.org/prj/imapsync_list/msg01756.html -http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-1/ -http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-2/ - - ======================================================================= Q. from Microsoft's Exchange 2007 to Google Apps for your Domain (GAFYD) @@ -142,37 +130,15 @@ I encounter many errors like this: R. GroupWise 7 seems buggy. Apply GroupWise 7 support pack 1 -======================================================================= -Q. Migrating from David Tobit V10 (DvISE Mail Access Server MA-...) - -R. Use the following options: - -imapsync ... --prefix1 "" --sep1 / --idatefromheader ^ - --nofoldersizes --useuid --nocheckmessageexists ======================================================================= -Q. Migrating from David Tobit V8 - ("* OK IMAP4rev1 DvISE Mail Access Server MA-8.10a (0126)") +Q. Migrating from or to David Tobit -First try above V10 solution since improvments have been made -to support Tobit. - -R. Use the following options : -imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed +R. See FAQ.David_Tobit.txt + https://imapsync.lamiral.info/FAQ.d/FAQ.David_Tobit.txt ======================================================================= -Q. Migrating from Tobit David Server 6 - ("DvISE Mail Access Server MA-6.60a (0118)") -First try above V10 solution since improvments have been made -to support Tobit. - -R. Look at the discussion: -http://www.linux-france.org/prj/imapsync_list/msg00582.html -http://www.linux-france.org/prj/imapsync_list/threads.html#00582 -patch saved in ./patches/imapsync-1.337_tobit_V6.patch - -======================================================================= Q. I need to migrate 1250 mailboxes, passwords are in a MySQL Database. Can you tell me if your script suits my needs? diff --git a/FAQ.d/FAQ.Zimbra.txt b/FAQ.d/FAQ.Zimbra.txt index 549a578..0f0ad31 100644 --- a/FAQ.d/FAQ.Zimbra.txt +++ b/FAQ.d/FAQ.Zimbra.txt @@ -1,5 +1,5 @@ #!/bin/cat -$Id: FAQ.Zimbra.txt,v 1.4 2019/01/28 22:39:28 gilles Exp gilles $ +$Id: FAQ.Zimbra.txt,v 1.6 2019/06/25 00:15:16 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ @@ -16,6 +16,27 @@ It's obsolete and it will give you, and me, more difficulties than doing a standard sync without additional options. +======================================================================= +Q. From Zimbra to XXX + + imapsync ... \ + --exclude "Conversation Action Settings" \ + --exclude "Quick Step Settings" \ + --exclude "News Feed" + +======================================================================= +Q. Zimbra disconnects without reason + Could not select: * BYE ... Zimbra IMAP4rev1 server closing connection + +R. The cause of this problem may be that some users have a subscription + of a shared folder from a mailbox locked and so the shared folder. + The access problems may happen in a lot of folders, + not only the shared/locked one. + + It should work after removing the locked/shared folder. + + Thanks to Danny Herrlitz for this input! + ======================================================================= Q. How to migrate from Zimbra with an admin account? diff --git a/FAQ.d/htaccess.txt b/FAQ.d/htaccess.txt index 647f785..aa4f041 100644 --- a/FAQ.d/htaccess.txt +++ b/FAQ.d/htaccess.txt @@ -1,5 +1,5 @@ -# $Id: htaccess.txt,v 1.23 2019/02/16 22:42:25 gilles Exp gilles $ +# $Id: htaccess.txt,v 1.25 2019/05/17 10:16:00 gilles Exp gilles $ AddDescription "Back to Imapsync main page." .. @@ -11,6 +11,7 @@ AddDescription "Big mailboxes tips." FAQ.Big_Mailb AddDescription "Connection issues." FAQ.Connection.txt AddDescription "Contacts & Calendars & Chat issues." FAQ.Contacts_Calendars.txt AddDescription "Dates issues." FAQ.Dates.txt +AddDescription "David Tobit." FAQ.David_Tobit.txt AddDescription "Domino." FAQ.Domino.txt AddDescription "Dovecot." FAQ.Dovecot.txt AddDescription "Duplicated messages issues." FAQ.Duplicates.txt @@ -33,6 +34,7 @@ AddDescription "Password & special characters on Windows." FAQ.Passwords AddDescription "Password & special characters on Unix." FAQ.Passwords_on_Unix.txt AddDescription "POP3 issues." FAQ.POP3.txt AddDescription "Selecting messages." FAQ.Messages_Selection.txt +AddDescription "How to transfer too big messages." FAQ.Messages_Too_Big.txt AddDescription "Oracle-UCS." FAQ.Oracle-UCS.txt AddDescription "Checklist before release a new release." FAQ.Release_Checklist.txt AddDescription "Guidelines to report bugs." FAQ.Reporting_Bugs.txt diff --git a/INSTALL.d/Dockerfile b/INSTALL.d/Dockerfile index e39c871..87027c7 100644 --- a/INSTALL.d/Dockerfile +++ b/INSTALL.d/Dockerfile @@ -1,6 +1,6 @@ ## Dockerfile for building a docker imapsync image -# $Id: Dockerfile,v 1.14 2018/09/16 10:42:11 gilles Exp gilles $ +# $Id: Dockerfile,v 1.15 2019/02/26 11:39:31 gilles Exp gilles $ # I use the following command to build the image: # # docker build -t gilleslamiral/imapsync . @@ -20,7 +20,8 @@ LABEL maintainer "gilles@lamiral.info" # Put a copy of the Dockerfile in the image itself # It can help future maintenance, isn't it? -COPY Dockerfile / +# Also put my last imapsync on / for testing purpose +COPY Dockerfile imapsync / RUN apt-get update \ && apt-get install -y \ diff --git a/INSTALL.d/INSTALL.Centos.txt b/INSTALL.d/INSTALL.Centos.txt index 2a0f878..3c5c83b 100644 --- a/INSTALL.d/INSTALL.Centos.txt +++ b/INSTALL.d/INSTALL.Centos.txt @@ -1,5 +1,5 @@ #!/bin/cat -# $Id: INSTALL.Centos.txt,v 1.10 2019/02/05 18:20:42 gilles Exp gilles $ +# $Id: INSTALL.Centos.txt,v 1.12 2019/05/02 12:41:07 gilles Exp gilles $ This documentation is also located online at https://imapsync.lamiral.info/INSTALL.d/ @@ -65,18 +65,28 @@ Then: perl-Test-Fatal \ perl-Test-Mock-Guard \ perl-Test-Requires \ - perl-Test-Deep - - wget https://imapsync.lamiral.info/imapsync + perl-Test-Deep \ + perl-File-Tail + wget -N https://imapsync.lamiral.info/imapsync chmod +x imapsync ./imapsync --testslive -Now you have the latest imapsync on Centos 7. +Now you should shave the latest imapsync locally working on Centos 7. + +If you want to install it and replace the olf one: + + mv /usr/bin/imapsync /usr/bin/imapsync_old + cp ./imapsync /usr/bin/imapsync + + imapsync --version + +should give the latest release number instead of 1.727 +(1.937 at the time of this writing) If you want to install an online imapsync service -like /X then you also need a recent CGI.pm, -add with cpanm: +like /X then you also need a recent decent CGI.pm, +add it with cpanm: cpanm CGI @@ -134,7 +144,8 @@ Then: perl-Test-Fatal \ perl-Test-Mock-Guard \ perl-Test-Requires \ - perl-Test-Deep + perl-Test-Deep \ + perl-File-Tail wget https://imapsync.lamiral.info/imapsync diff --git a/INSTALL.d/INSTALL.OnlineUI.txt b/INSTALL.d/INSTALL.OnlineUI.txt index b8cbbd6..61b05ca 100644 --- a/INSTALL.d/INSTALL.OnlineUI.txt +++ b/INSTALL.d/INSTALL.OnlineUI.txt @@ -1,5 +1,5 @@ #!/bin/cat -# $Id: INSTALL.OnlineUI.txt,v 1.29 2019/02/18 09:24:18 gilles Exp gilles $ +# $Id: INSTALL.OnlineUI.txt,v 1.30 2019/03/06 17:36:50 gilles Exp gilles $ This documentation is also located online at https://imapsync.lamiral.info/INSTALL.d/ @@ -73,6 +73,13 @@ Print the CGI.pm release with: perl -MCGI -e 'print "$CGI::VERSION\n"' +If it is under release 4.08 (2014-10-18) then upgrade it with + + cpanm CGI + +It is a good thing to remove the old one if it was installed +by a distribution package, I let you this part as an exercise. + To check and fix the Perl modules dependencies, run: cd diff --git a/INSTALL.d/prerequisites_imapsync b/INSTALL.d/prerequisites_imapsync index 9e6869d..c06fef3 100755 --- a/INSTALL.d/prerequisites_imapsync +++ b/INSTALL.d/prerequisites_imapsync @@ -1,8 +1,9 @@ #!/bin/sh -# $Id: prerequisites_imapsync,v 1.29 2018/09/28 14:16:44 gilles Exp gilles $ +# $Id: prerequisites_imapsync,v 1.32 2019/05/27 18:52:23 gilles Exp gilles $ MODULES_MANDATORY=' +File::Tail App::cpanminus Authen::NTLM CGI @@ -144,7 +145,12 @@ search_modules_any() { return } - + # FreeBSD + pkg version > /dev/null 2>&1 && { + search_modules_freebsd "$@" + return + } + # no yum, no apt-get, no pacman { search_modules_cpanm "$@" @@ -202,16 +208,32 @@ search_modules_apt() { echo echo Searching deb packages names for M in "$@" ; do - echo "==== Searching deb package name for $M" F=`echo $M|tr -s ":" "/"`.pm + echo "==== Searching deb package name for $M with: apt-file search /$F" #echo apt-file search /$F echo apt-file search /$F + echo + echo "==== Searching deb package name for $M with: apt-cache search $M" #apt-cache search "$M" + apt-cache search "$M" echo done } +search_modules_freebsd() +{ + echo + echo Searching pkg FreeBSD names + for M in "$@" ; do + F=`echo $M|tr -s ":" "-"` + echo "==== Searching deb package name for $M with: pkg search p5-$F" + echo + pkg search "p5-$F" + echo + done + +} list_to_install() { test -n "$LIST_TO_INSTALL" || return 0 diff --git a/Makefile b/Makefile index 72c1685..79e4298 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ -# $Id: Makefile,v 1.290 2019/02/18 12:02:32 gilles Exp gilles $ +# $Id: Makefile,v 1.301 2019/05/28 14:25:35 gilles Exp gilles $ .PHONY: help usage all doc @@ -26,8 +26,9 @@ usage: @echo "make W/test_exe_2.bat # run W/test_exe_2.bat on win32" @echo "make examples/sync_loop_windows.bat # run examples/sync_loop_windows.bat on win32" - @echo "make win32_prereq # run W/install_modules.bat on win32" - @echo "make win32_update_ssl # run W/install_module_ssl.bat on win32" + @echo "make W/install_modules.bat # run W/install_modules.bat on win32" + @echo "make W/install_module_one.bat # run W/install_module_one.bat on win32" + @echo "make W/install_module_ssl.bat # run W/install_module_ssl.bat on win32" @echo "make all " @echo "make upload_tests # upload tests.sh" @echo "make upload_index" @@ -62,7 +63,7 @@ VERSION := $(shell perl ./imapsync --version 2>/dev/null || cat VERSION) VERSION_PREVIOUS := $(shell perl ./$(DIST_PATH)/imapsync --version 2>/dev/null || echo ERROR) VERSION_EXE := $(shell cat ./VERSION_EXE) -IMAPClient_3xx := ./W/Mail-IMAPClient-3.40/lib +IMAPClient_3xx := ./W/Mail-IMAPClient-3.42/lib IMAPClient := $(IMAPClient_3xx) @@ -186,7 +187,16 @@ install: testp W/imapsync.1 dev: test crit cover nytprof bin docker: - ssh ks3 'cd docker/imapsync && . memo' + ssh ks3 'cd docker/imapsync && . memo_docker' + @echo "make docker_copy_to_ks3 # copy imapsync Dockerfile memo_docker to ks3" + @echo "make docker_upload_docker_hub # upload last build to https://hub.docker.com/r/gilleslamiral/imapsync" + + +docker_copy_to_ks3: + rsync -av imapsync INSTALL.d/Dockerfile INSTALL.d/memo_docker ks3:docker/imapsync/ + +docker_upload_docker_hub: + ssh ks3 'cd docker/imapsync && . memo_docker && imapsync_docker_upload' nytprof: sh tests.sh ll_nytprof @@ -378,8 +388,6 @@ W/learn_func.bat: ssh Admin@c 'C:/msys/1.0/home/Admin/imapsync/learn_func.bat' ./W/check_winerr learn_func.bat -win32_prereq: W/install_modules.bat - W/install_modules.bat: unix2dos W/install_modules.bat scp W/install_modules.bat Admin@c:'C:/msys/1.0/home/Admin/imapsync/' @@ -387,7 +395,7 @@ W/install_modules.bat: ./W/check_winerr install_modules.bat -win32_update_ssl: +W/install_module_ssl.bat: scp W/install_module_ssl.bat Admin@c:'C:/msys/1.0/home/Admin/imapsync/' ssh Admin@c 'C:/msys/1.0/home/Admin/imapsync/install_module_ssl.bat' @@ -413,6 +421,7 @@ imapsync.exe: imapsync ./W/check_winerr test_exe.bat rm -f imapsync.exe scp Admin@c:'C:/msys/1.0/home/Admin/imapsync/imapsync.exe' . + chmod a+r+x imapsync.exe (date "+%s"| tr "\n" " "; echo -n "END " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME exe: imapsync @@ -423,20 +432,40 @@ exe: imapsync ssh Admin@c 'C:/msys/1.0/home/Admin/imapsync/imapsync.exe --justbanner' rm -f imapsync.exe scp Admin@c:'C:/msys/1.0/home/Admin/imapsync/imapsync.exe' . + chmod a+r+x imapsync.exe (date "+%s"| tr "\n" " "; echo -n "END " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME zzz: unix2dos W/build_exe.bat W/install_module_one.bat - scp imapsync W/build_exe.bat W/install_module_one.bat pc_HP_DV7_p24:'Desktop/imapsync_build' -# ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' -# ./W/check_win64err build_exe.bat + scp imapsync W/build_exe.bat W/install_module_one.bat W/test_exe_testsdebug.bat pc_HP_DV7_p24:'Desktop/imapsync_build' + ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' + ./W/check_win64err build_exe.bat + +zzz2: + unix2dos W/test_exe_testsdebug.bat W/test_exe_tests.bat + scp W/test_exe_testsdebug.bat W/test_exe_tests.bat pc_HP_DV7_p24:'Desktop/imapsync_build' + ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/test_exe_testsdebug.bat' + ./W/check_win64err test_exe_testsdebug.bat + ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/test_exe_tests.bat' + ./W/check_win64err test_exe_tests.bat + +W/test4.bat: + unix2dos W/test4.bat + scp W/test4.bat pc_HP_DV7_p24:'Desktop/imapsync_build' + ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/test4.bat' + +W/test5.bat: + unix2dos W/test5.bat + scp W/test5.bat pc_HP_DV7_p24:'Desktop/imapsync_build' + ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/test5.bat' + imapsync_64bit.exe: imapsync (date "+%s"| tr "\n" " "; echo -n "BEGIN 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME ssh 'pc HP DV7'@p24 'perl -V' unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \ - W/test_exe_tests.bat W/test_exe.bat \ + W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \ pc_HP_DV7_p24:'Desktop/imapsync_build' ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' ./W/check_win64err build_exe.bat @@ -444,6 +473,7 @@ imapsync_64bit.exe: imapsync ./W/check_win64err test_exe.bat rm -f imapsync_64bit.exe scp pc_HP_DV7_p24:'Desktop/imapsync_build/imapsync.exe' imapsync_64bit.exe + chmod a+r+x imapsync_64bit.exe (date "+%s"| tr "\n" " "; echo -n "END 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME @@ -486,6 +516,10 @@ mactestslive: rsync -p -e 'ssh -4 -p 995' imapsync gilleslamira@gate.polarhome.com: ssh -4 -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --testslive' +mactestslive6: + rsync -p -e 'ssh -4 -p 995' imapsync gilleslamira@gate.polarhome.com: + ssh -4 -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --testslive6' + bin: lin mac win win64 @@ -510,7 +544,6 @@ $(BIN_NAME): imapsync ./$(BIN_NAME) --justbanner - lfo: upload_lfo .PHONY: tarball cidone ci @@ -529,6 +562,7 @@ ci: cidone cidone: rcsdiff RCS/* + rcsdiff X/cgi_memo X/stat_patterns.txt X/imapsync_form.html X/imapsync_form.js X/imapsync_form.css X/noscript.css rcsdiff W/*.bat W/*.sh W/*.txt W/*.txt W/*.htaccess cd W && rcsdiff RCS/* rcsdiff doc/*.t2t @@ -537,7 +571,7 @@ cidone: rcsdiff examples/*.sh examples/*.bat examples/*.txt cd examples && rcsdiff RCS/* rcsdiff W/tools/backup_old_dist W/tools/gen_README_dist W/tools/validate_html4 W/tools/validate_xml_html5 W/tools/fix_email_for_exchange.py - rcsdiff S/*.txt S/*.shtml S/*.html + rcsdiff S/*.txt S/*.shtml S/*.html dist: cidone test clean all perlcritic dist_prepa dist_zip README_dist @@ -616,7 +650,7 @@ ks2tests_root: ssh root@ks.lamiral.info './imapsync --tests' i005tests_root: - rsync -P imapsync root@i005.lamiral.info: + rsync -P imapsync INSTALL.d/prerequisites_imapsync root@i005.lamiral.info: ssh root@i005.lamiral.info './imapsync --tests' ks2testslive: @@ -703,7 +737,7 @@ ci_imapsync: rcsdiff imapsync upload_latest: unitests ci_imapsync bin - rsync -a imapsync imapsync_bin_Linux_i686 imapsync_bin_Darwin imapsync.exe ./INSTALL.d/prerequisites_imapsync ../imapsync_website/ + rsync -a imapsync imapsync_bin_Linux_i686 imapsync_bin_Darwin imapsync.exe imapsync_64bit.exe ./INSTALL.d/prerequisites_imapsync ../imapsync_website/ rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' diff --git a/README b/README index c6a1602..4eb395b 100644 --- a/README +++ b/README @@ -5,7 +5,7 @@ NAME VERSION - This documentation refers to Imapsync $Revision: 1.921 $ + This documentation refers to Imapsync $Revision: 1.945 $ USAGE @@ -25,44 +25,59 @@ DESCRIPTION another. Imapsync command is a tool allowing incremental and recursive imap - transfers from one mailbox to another. + transfers from one mailbox to another. If you don't understand the + previous sentence, it's normal, it's pedantic computer oriented jargon. - By default all folders are transferred, recursively, meaning the whole - folder hierarchy is taken, all messages in them, and all messages flags - (\Seen \Answered \Flagged etc.) are synced too. + All folders are transferred, recursively, meaning the whole folder + hierarchy is taken, all messages in them, and all messages flags (\Seen + \Answered \Flagged etc.) are synced too. Imapsync reduces the amount of data transferred by not transferring a - given message if it resides already on both sides. + given message if it already resides on the destination side. Messages + that are on the destination side but not on the source side stay as they + are (see the --delete2 option to have a strict sync). - Same specific headers and the transfer is done only once. By default, - the identification headers are "Message-Id:" and "Received:" lines but - this choice can be changed with the --useheader option. + How imapsync knows a message is already on both sides? Same specific + headers and the transfer is done only once. By default, the + identification headers are "Message-Id:" and "Received:" lines but this + choice can be changed with the --useheader option. All flags are preserved, unread messages will stay unread, read ones will stay read, deleted will stay deleted. - You can stop the transfer at any time and restart it later, imapsync - works well with bad connections and interruptions, by design. + You can abort the transfer at any time and restart it later, imapsync + works well with bad connections and interruptions, by design. On a + terminal hit Ctr-c twice within two seconds in order to abort the + program. Hit Ctr-c just once makes imapsync reconnect to both imap + servers. - You can decide to delete the messages from the source mailbox after a - successful transfer, it can be a good feature when migrating live - mailboxes since messages will be only on one side. - - In that case, use the --delete1 option. Option --delete1 implies also - option --expunge1 so all messages marked deleted on host1 will be really - deleted. - - You can also decide to remove empty folders once all of their messages - have been transferred. Add --delete1emptyfolders to obtain this - behavior. - - A different scenario is synchronizing a mailbox B from another mailbox A - in case you just want to keep a "live" copy of A in B. + A classical scenario is synchronizing a mailbox B from another mailbox A + in case you just want to keep a strict copy of A in B. Strict meaning + all messages in A will be in B but no more. For this, option --delete2 has to be used, it deletes messages in host2 folder B that are not in host1 folder A. If you also need to destroy host2 folders that are not in host1 then use --delete2folders. See also - --delete2foldersonly and --delete2foldersbutnot. + --delete2foldersonly and --delete2foldersbutnot to set up exceptions on + folders to destroy (INBOX will never be destroy, it's a mandatory folder + in IMAP). + + A different scenario is to delete the messages from the source mailbox + after a successful transfer, it can be a good feature when migrating + mailboxes since messages will be only on one side. The source account + will only have messages that are not on the destination yet, ie, + messages that arrived after a sync or that failed to be copied. + + In that case, use the --delete1 option. Option --delete1 implies also + option --expunge1 so all messages marked deleted on host1 will be really + deleted. In IMAP protocol deleting a message does not really delete it, + it marks it with the flag \Deleted, allowing an undelete. Expunging a + folder removes, definitively, all the messages marked as \Deleted in + this folder. + + You can also decide to remove empty folders once all of their messages + have been transferred. Add --delete1emptyfolders to obtain this + behavior. Imapsync is not adequate for maintaining two active imap accounts in synchronization when the user plays independently on both sides. Use @@ -73,9 +88,9 @@ OPTIONS usage: imapsync [options] - Mandatory options are the six values, three on each sides, needed to log - in into the IMAP servers, ie, a host, a username, and a password, two - times. + Standard options are the six values forming the credentials, three on + each sides, needed to log in into the IMAP servers, ie, a host, a + username, and a password, two times. Conventions used: @@ -89,26 +104,28 @@ OPTIONS OPTIONS/credentials - --host1 str : Source or "from" imap server. Mandatory. + --host1 str : Source or "from" imap server. --port1 int : Port to connect on host1. - Optional since default port is 143 or 993 if --ssl1 - --user1 str : User to login on host1. Mandatory. + Optional since default ports are the + well known ports 143 or 993. + --user1 str : User to login on host1. --password1 str : Password for the user1. - --host2 str : "destination" imap server. Mandatory. - --port2 int : Port to connect on host2. - Optional since default port is 143 or 993 if --ssl2 - --user2 str : User to login on host2. Mandatory. + --host2 str : "destination" imap server. + --port2 int : Port to connect on host2. Optional + --user2 str : User to login on host2. --password2 str : Password for the user2. --showpasswords : Shows passwords on output instead of "MASKED". - Useful to restart a complete run by just reading the log, - or to debug passwords. It's not a secure practice. + Useful to restart a complete run by just reading + the command line used in the log, + or to debug passwords. + It's not a secure practice. --passfile1 str : Password file for the user1. It must contain the - password on the first line. This option avoids to show + password on the first line. This option avoids showing the password on the command line like --password1 does. - --passfile2 str : Password file for the user2. Contains the password. + --passfile2 str : Password file for the user2. You can also pass the passwords in the environment variables IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2 @@ -174,8 +191,9 @@ OPTIONS --nomixfolders : Do not merge folders when host1 is case-sensitive while host2 is not (like Exchange). Only the first - similar folder is synced (ex: with Sent SENT sent - on host1 only Sent will be synced to host2). + similar folder is synced (example: with folders + "Sent", "SENT" and "sent" + on host1 only "Sent" will be synced to host2). --skipemptyfolders : Empty host1 folders are not created on host2. @@ -196,9 +214,20 @@ OPTIONS --subfolder2 str : Syncs the whole host1 folders hierarchy under the host2 folder named str. - (It does it internally by adding two + It does it internally by adding three --regextrans2 options before all others. - Add --debug to see what's really going on.) + Add --debug to see what's really going on. + + --subfolder1 str : Syncs the host1 folders hierarchy under str + to the root hierarchy of host2. + It's the couterpart of a sync done by --subfolder2 + when doing it in the reverse order. + Backup/Restore scenario: + Use --subfolder2 str for a backup to the folder str + on host2. Then use --subfolder1 str for restoring + from the folder str, after inverting + host1/host2 user1/user2 values. + --subscribed : Transfers subscribed folders. --subscribe : Subscribe to the folders transferred on the @@ -207,21 +236,29 @@ OPTIONS host2 even if they are not subscribed on host1. --prefix1 str : Remove prefix str to all destination folders, - usually INBOX. or INBOX/ or an empty string "". + usually "INBOX." or "INBOX/" or an empty string "". imapsync guesses the prefix if host1 imap server - does not have NAMESPACE capability. This option + does not have NAMESPACE capability. So this option should not be used, most of the time. --prefix2 str : Add prefix to all host2 folders. See --prefix1 - --sep1 str : Host1 separator in case NAMESPACE is not supported. - --sep2 str : Host2 separator in case NAMESPACE is not supported. + + --sep1 str : Host1 separator. This option should not be used, + most of the time. + Imapsync gets the separator from the server itself, + by using NAMESPACE, or it tries to guess it + from the folders listing (it counts + characters / . \\ \ in folder names and choose the + more frequent, or finally / if nothing is found. + --sep2 str : Host2 separator. --regextrans2 reg : Apply the whole regex to each destination folders. --regextrans2 reg : and this one. etc. When you play with the --regextrans2 option, first add also the safe options --dry --justfolders Then, when happy, remove --dry, remove --justfolders. - Have in mind that --regextrans2 is applied after prefix - and separator inversion. For examples see + Have in mind that --regextrans2 is applied after + the automatic prefix and separator inversion. + For examples see: https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt OPTIONS/folders sizes @@ -260,7 +297,14 @@ OPTIONS --pipemess cmd : Apply this cmd command to each message content before the copy. - --pipemess cmd : and this one, etc. + --pipemess cmd : and this one, etc. + With several --pipemess, the output of each cmd + command (STDOUT) is given to the input (STDIN) + of the next command. + For example, + --pipemess cmd1 --pipemess cmd2 --pipemess cmd3 + is like a Unix pipe: + "cat message | cmd1 | cmd2 | cmd3" --disarmreadreceipts : Disarms read receipts (host2 Exchange issue) @@ -270,6 +314,9 @@ OPTIONS OPTIONS/flags + If you encounter flag problems see also: + https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt + --regexflag reg : Apply the whole regex to each flags list. Example: 's/"Junk"//g' # to remove "Junk" flag. --regexflag reg : then this one, etc. @@ -288,19 +335,26 @@ OPTIONS \Deleted, then messages are really deleted with an EXPUNGE IMAP command. If expunging after each message slows down too much the sync then use - --noexpungeaftereach to speed up. + --noexpungeaftereach to speed up, expunging will then be + done only twice per folder, one at the beginning and + one at the end of a folder sync. + --expunge1 : Expunge messages on host1 just before syncing a folder. Expunge is done per folder. Expunge aims is to really delete messages marked deleted. An expunge is also done after each message copied - if option --delete1 is set. + if option --delete1 is set (unless --noexpungeaftereach). + --noexpunge1 : Do not expunge messages on host1. + --delete1emptyfolders : Deletes empty folders on host1, INBOX excepted. Useful with --delete1 since what remains on host1 is only what failed to be synced. --delete2 : Delete messages in host2 that are not in host1 server. Useful for backup or pre-sync. + --delete2 implies --uidexpunge2 + --delete2duplicates : Delete messages in host2 that are duplicates. Works only without --useuid since duplicates are detected with an header part of each message. @@ -308,30 +362,33 @@ OPTIONS --delete2folders : Delete folders in host2 that are not in host1 server. For safety, first try it like this (it is safe): --delete2folders --dry --justfolders --nofoldersizes + --delete2foldersonly reg : Deleted only folders matching regex. Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/" + --delete2foldersbutnot reg : Do not delete folders matching regex. Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/" - --expunge2 : Expunge messages on host2 after messages transfer. - --uidexpunge2 : uidexpunge messages on the host2 account - that are not on the host1 account, requires --delete2 + --noexpunge2 : Do not expunge messages on host2. + --nouidexpunge2 : Do not uidexpunge messages on the host2 account + that are not on the host1 account. OPTIONS/dates + If you encounter problems with dates, see also: + https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt + --syncinternaldates : Sets the internal dates on host2 same as host1. Turned on by default. Internal date is the date - a message arrived on a host (mtime). + a message arrived on a host (Unix mtime). --idatefromheader : Sets the internal dates on host2 same as the - "Date:" headers. - If you encounter problems with dates see also - https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt + ones in "Date:" headers. OPTIONS/message selection --maxsize int : Skip messages larger (or equal) than int bytes --minsize int : Skip messages smaller (or equal) than int bytes - --maxage int : Skip messages older than int days. + --maxage int : Skip messages older than int days. final stats (skipped) don't count older messages see also --minage --minage int : Skip messages newer than int days. @@ -362,7 +419,7 @@ OPTIONS --usecache : Use cache to speed up the sync. --nousecache : Do not use cache. Caveat: --useuid --nousecache creates duplicates on multiple runs. - --useuid : Use UIDs instead of headers as a criterium to recognize + --useuid : Use UIDs instead of headers as a criterion to recognize messages. Option --usecache is then implied unless --nousecache is used. @@ -370,8 +427,11 @@ OPTIONS --syncacls : Synchronizes acls (Access Control Lists). --nosyncacls : Does not synchronize acls. This is the default. - Acls in IMAP are not standardized, be careful. - --addheader : When a message has no headers to be identified, + Acls in IMAP are not standardized, be careful + since one acl code on one side may signify something + else on the other one. + + --addheader : When a message has no headers to be identified, --addheader adds a "Message-Id" header, like "Message-Id: 12345@imapsync", where 12345 is the imap UID of the message on the host1 folder. @@ -423,16 +483,25 @@ OPTIONS --abort : terminates a previous call still running. It uses the pidfile to know what process to abort. - --exitwhenover int : Stop syncing when total bytes transferred reached. + --exitwhenover int : Stop syncing and exits when int total bytes + transferred is reached. --version : Print only software version. - --noreleasecheck : Do not check for new imapsync release (a http request). - --releasecheck : Check for new imapsync release (a http request). + --noreleasecheck : Do not check for new imapsync release + --releasecheck : Check for new imapsync release. + it's an http request to + http://imapsync.lamiral.info/prj/imapsync/VERSION + --noid : Do not send/receive ID command to imap servers. + --justconnect : Just connect to both servers and print useful information. Need only --host1 and --host2 options. + Obsolete since "imapsync --host1 imaphost" alone + implies --justconnect + --justlogin : Just login to both host1 and host2 with users credentials, then exit. + --justfolders : Do only things about folders (ignore messages). --help : print this help. @@ -449,18 +518,21 @@ OPTIONS SECURITY You can use --passfile1 instead of --password1 to give the password - since it is safer. With --password1 option, any user on your host can - see the password by using the 'ps auxwwww' command. Using a variable - (like $PASSWORD1) is also dangerous because of the 'ps auxwwwwe' - command. So, saving the password in a well protected file (600 or - rw-------) is the best solution. + since it is safer. With --password1 option, on Linux, any user on your + host can see the password by using the 'ps auxwwww' command. Using a + variable (like IMAPSYNC_PASSWORD1) is also dangerous because of the 'ps + auxwwwwe' command. So, saving the password in a well protected file (600 + or rw-------) is the best solution. - Imapsync activates ssl or tls encryption by default, if possible. What - detailed behavior is under this "if possible"? Imapsync activates ssl if - the well known port imaps port (993) is open on the imap servers. If the - imaps port is closed then it open a normal (clear) connection on port - 143 but it looks for TLS support in the CAPABILITY list of the servers. - If TLS is supported then imapsync goes to encryption. + Imapsync activates ssl or tls encryption by default, if possible. + + What detailed behavior is under this "if possible"? + + Imapsync activates ssl if the well known port imaps port (993) is open + on the imap servers. If the imaps port is closed then it open a normal + (clear) connection on port 143 but it looks for TLS support in the + CAPABILITY list of the servers. If TLS is supported then imapsync goes + to encryption. If the automatic ssl/tls detection fails then imapsync will not protect against sniffing activities on the network, especially for passwords. @@ -473,7 +545,25 @@ SECURITY EXIT STATUS Imapsync will exit with a 0 status (return code) if everything went - good. Otherwise, it exits with a non-zero status. + good. Otherwise, it exits with a non-zero status. Here is the list of + the exit code values (an integer between 0 and 255), the names reflects + their meaning: + + EX_OK => 0 ; #/* successful termination */ + EX_USAGE => 64 ; #/* command line usage error */ + EX_NOINPUT => 66 ; #/* cannot open input */ + EX_UNAVAILABLE => 69 ; #/* service unavailable */ + EX_SOFTWARE => 70 ; #/* internal software error */ + EXIT_CATCH_ALL => 1 ; # Any other error + EXIT_BY_SIGNAL => 6 ; # Should be 128+n where n is the sig_num + EXIT_PID_FILE_ERROR => 8 ; + EXIT_CONNECTION_FAILURE => 10 ; + EXIT_TLS_FAILURE => 12 ; + EXIT_AUTHENTICATION_FAILURE => 16 ; + EXIT_SUBFOLDER1_NO_EXISTS => 21 ; + EXIT_WITH_ERRORS => 111 ; + EXIT_WITH_ERRORS_MAX => 112 ; + EXIT_TESTS_FAILED => 254 ; # Like Test::More API LICENSE AND COPYRIGHT @@ -488,13 +578,14 @@ LICENSE AND COPYRIGHT "No limits to do anything with this work and this license." - https://imapsync.lamiral.info/LICENSE + Look at https://imapsync.lamiral.info/LICENSE AUTHOR Gilles LAMIRAL - Feedback good or bad is very often welcome. + Good feedback good is always welcome. Bad feedback is very often + welcome. Gilles LAMIRAL earns his living by writing, installing, configuring and teaching free, open and often gratis software. Imapsync used to be @@ -512,10 +603,6 @@ IMAP SERVERS supported HUGE MIGRATION - Pay special attention to options --subscribed --subscribe --delete1 - --delete1emptyfolders --delete2 --delete2folders --maxage --minage - --maxsize --useuid --usecache - If you have many mailboxes to migrate think about a little shell program. Write a file called file.txt (for example) containing users and passwords. The separator used in this example is ';' @@ -549,10 +636,10 @@ HUGE MIGRATION INSTALL - Imapsync works under any Unix with perl. + Imapsync works under any Unix with Perl. Imapsync works under most Windows (2000, XP, Vista, Seven, Eight, Ten - and all Server releases 2000, 2003, 2008 and R2, 2012 and R2) + and all Server releases 2000, 2003, 2008 and R2, 2012 and R2, 2016) as a standalone binary software called imapsync.exe, usually launched from a batch file in order to avoid always typing the options. @@ -590,37 +677,54 @@ SIMILAR SOFTWARE See also https://imapsync.lamiral.info/S/external.shtml for a better up to date list. - imap_tools : https://github.com/andrewnimmo/rick-sanders-imap-tools - offlineimap : https://github.com/nicolas33/offlineimap - Doveadm-Sync : http://wiki2.dovecot.org/Tools/Doveadm/Sync - ( Dovecot sync tool ) - mbsync : http://isync.sourceforge.net/ - mailsync : http://mailsync.sourceforge.net/ - mailutil : http://www.washington.edu/imap/ - part of the UW IMAP tookit. - imaprepl : http://www.bl0rg.net/software/ - http://freecode.com/projects/imap-repl/ - imapcopy : http://www.ardiehl.de/imapcopy/ - migrationtool : http://sourceforge.net/projects/migrationtool/ - imapmigrate : http://sourceforge.net/projects/cyrus-utils/ - wonko_imapsync: http://wonko.com/article/554 - see also file W/tools/wonko_ruby_imapsync - exchange-away : http://exchange-away.sourceforge.net/ - pop2imap : http://www.linux-france.org/prj/pop2imap/ + Last updated and verified on Thu Apr 11, 2019. - Feedback (good or bad) will often be welcome. + imapsync : https://github.com/imapsync/imapsync + (this is an imapsync copy, sometimes delayed, + with --noreleasecheck by default since release 1.592, 2014/05/22) + imap_tools : https://web.archive.org/web/20161228145952/http://www.athensfbc.com/imap_tools/ + The imap_tools code is now at + https://github.com/andrewnimmo/rick-sanders-imap-tools + imaputils : https://github.com/mtsatsenko/imaputils (very old imap_tools fork) + Doveadm-Sync : https://wiki2.dovecot.org/Tools/Doveadm/Sync ( Dovecot sync tool ) + davmail : http://davmail.sourceforge.net/ + offlineimap : http://offlineimap.org/ + mbsync : http://isync.sourceforge.net/ + mailsync : http://mailsync.sourceforge.net/ + mailutil : http://www.washington.edu/imap/ part of the UW IMAP tookit. + imaprepl : https://bl0rg.net/software/ http://freecode.com/projects/imap-repl/ + imapcopy (Pascal): http://www.ardiehl.de/imapcopy/ + imapcopy (Java) : https://code.google.com/archive/p/imapcopy/ + imapsize : http://www.broobles.com/imapsize/ + migrationtool : http://sourceforge.net/projects/migrationtool/ + imapmigrate : http://sourceforge.net/projects/cyrus-utils/ + larch : https://github.com/rgrove/larch (derived from wonko_imapsync, good at Gmail) + wonko_imapsync : http://wonko.com/article/554 (superseded by larch) + pop2imap : http://www.linux-france.org/prj/pop2imap/ (I wrote that too) + exchange-away : http://exchange-away.sourceforge.net/ + SyncBackPro : http://www.2brightsparks.com/syncback/sbpro.html + ImapSyncClient : https://github.com/ridaamirini/ImapSyncClient + MailStore : https://www.mailstore.com/en/products/mailstore-home/ + mnIMAPSync : https://github.com/manusa/mnIMAPSync + imap-upload : http://imap-upload.sourceforge.net/ + (a tool for uploading a local mbox file to IMAP4 server) HISTORY - I wrote imapsync because an enterprise (basystemes) paid me to install a - new imap server without losing huge old mailboxes located in a far away - remote imap server, accessible by a low-bandwidth often broken link. The - tool imapcp (written in python) could not help me because I had to - verify every mailbox was well transferred, and then delete it after a - good transfer. Imapsync started its life as a patch of the - copy_folder.pl script. The script copy_folder.pl comes from the - Mail-IMAPClient-2.1.3 perl module tarball source (more precisely in the - examples/ directory of the Mail-IMAPClient tarball). So many happened - since then that I wonder if it remains any lines of the original - copy_folder.pl in imapsync source code. + I initially wrote imapsync in July 2001 because an enterprise, + basystemes, paid me to install a new imap server without losing huge old + mailboxes located in a far away remote imap server, accessible by an + often broken low-bandwidth ISDN link. + + I had to verify every mailbox was well transferred, all folders, all + messages, without wasting bandwidth or creating duplicates upon resyncs. + The design was made with the beautiful rsync command in mind. + + Imapsync started its life as a patch of the copy_folder.pl script. The + script copy_folder.pl comes from the Mail-IMAPClient-2.1.3 perl module + tarball source (more precisely in the examples/ directory of the + Mail-IMAPClient tarball). + + So many happened since then that I wonder if it remains any lines of the + original copy_folder.pl in imapsync source code. diff --git a/README.txt b/README.txt new file mode 120000 index 0000000..100b938 --- /dev/null +++ b/README.txt @@ -0,0 +1 @@ +README \ No newline at end of file diff --git a/README_Windows.txt b/README_Windows.txt index 6865bea..5f27de1 100644 --- a/README_Windows.txt +++ b/README_Windows.txt @@ -1,4 +1,4 @@ -# $Id: README_Windows.txt,v 1.13 2019/02/14 15:50:09 gilles Exp gilles $ +# $Id: README_Windows.txt,v 1.14 2019/06/26 22:32:28 gilles Exp gilles $ # # This is the README_Windows.txt file for imapsync # imapsync : IMAP sync and migrate tool. @@ -38,19 +38,23 @@ In the folder extracted imapsync_1.xxx you see 6 files and 2 directories: * imapsync_example.bat is a simple batch file example you will copy and edit. * sync_loop_windows.bat is a batch file example for syncing many accounts. * file.txt is an input file example for syncing many accounts. -* imapsync.exe is the imapsync binary. You don't have to run it directly. +* imapsync.exe is the imapsync 32bit binary. You don't have to run it directly. +* imapsync_64bit.exe is the imapsync 64bit binary. You don't have to run it directly. * Cook/ is the directory to build imapsync.exe from its source, for the B) way and expert users. You can copy or rename the file imapsync_example.bat as you wish, -as long as its extension remains ".bat". +as long as its extension remains ".bat". + On Windows systems .bat extension means "I'm a batch script". -Same thing for sync_loop_windows.bat. +It's the same for sync_loop_windows.bat. + The batch scripts have to stay in the same directory than -imapsync.exe because of the way they call imapsync.exe, -they use ".\imapsync.exe", so let them be in the same directory. -Or change the path .\ to whatever you want if you understand what -you're doing (you have to use a pathname from the script point of view). +imapsync.exe because of the way they call imapsync.exe. +They use ".\imapsync.exe", so let them be in the same directory. + +Or you can change the path .\ to whatever you want if you understand what +you are doing (you have to use a pathname from the script point of view). For the rest of this documentation I assume you copied imapsync_example.bat to a file named imapsync_stuff.bat diff --git a/S/external.shtml b/S/external.shtml index 4323cc3..4837386 100644 --- a/S/external.shtml +++ b/S/external.shtml @@ -26,13 +26,13 @@ -

Similar software (back to menu) +

Similar software tools (back to menu)

External online IMAP migration services (back to menu)

I have no share in these free or paid services. -I don't think they use Imapsync. -Prices are given par mailbox and may be outdated (December 2011).

+Some use Imapsync, most don't. +Prices are given par mailbox and may be outdated +(Last checked on Thu Apr 11, 2019).

+