Merge pull request #284 from florenthemmi/update-faq.d

Update FAQ.d

FAQ.d/FAQ.APPEND_errors.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.APPEND_errors.txt,v 1.6 2019/02/16 22:38:49 gilles Exp gilles $
+$Id: FAQ.APPEND_errors.txt,v 1.12 2021/02/10 19:42:42 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,15 +11,71 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.APPEND_errors.txt
 
 Questions answered in this FAQ are:
 
+
+Q. I have an "could not append" error with "Message contains invalid header" at
+   the end, like:
+   Err 1/20: - msg INBOX/6 {40666} could not append ... NO Message contains invalid header
+   What can I do to transfer these emails?
+
 Q. For some messages, the imapsync log says 
    "could not append", sometimes followed by an explicit message 
    describing what went wrong, or sometimes followed by a not very
    useful message "socket closed while reading data from server"
    What can I do?
 
-R0. Well, the problem is that the "socket closed ..." error message happens
+
-    in several different issues. So I list here several potential issues
+Now the questions again with their answers.
+
+=======================================================================
+Q. I have an "could not append" error with "Message contains invalid header" 
+   at the end, like:
+   Err 1/20: - msg INBOX/6 {40666} could not append ... NO Message contains invalid header
+   What can I do to transfer these emails?
+
+R0. Append in the IMAP protocol is the command to add a message in the folder.
+    The error string "NO. Message contains invalid header" comes from the 
+    destination IMAP server at host2, it doesn't like the message and rejects it.
+   
+R1. Header lines must be in 7bit encoding.
+   If they aren't in 7bit the IMAP server can refuse them as they break 
+   the IMAP RFC specification.
+
+   A solution is to transform all 8bit characters to the X character with:
+   
+   imapsync ... --regexmess 'tr [\x80-\xff] [X]'
+
+   Caveat: this transformation transforms both the email header and its body.
+   It's not perfect but I guess it's better than no message transferred.
+
+R2. Some crappy email systems, like virus checkers or any other crappy 
+    software tool dealing with your messages, can add headers to tell 
+    the world that they've done a great job. But sometimes they didn't,
+    and they fucked up your messages by not respecting the standard
+    and added ugly non-rfc compliant headers.
+    
+    To remove an ugly header, let's call it "X-Spam-Report", that spreads over 
+    several lines beginning or, that's the ugly part, not beginning with a space:
+
+    imapsync ... --regexmess 's{X-Spam-Report:.*?\n(^[a-zA-Z0-9\-]+:|^\r?\n)}{$1}xms'
+
+    Thanks to Damien SAUTEREAU for reporting and solving this issue.
+    
+
+=======================================================================
+Q. For some messages, the imapsync log says 
+   "could not append", sometimes followed by an explicit message 
+   describing what went wrong, or sometimes followed by a not very
+   useful message "socket closed while reading data from server"
+   What can I do?
+
+R0. Append errors are host2 problems, append is the imap term to copy 
+    a message to the destination account. 
+    A problem is that the assosiated "socket closed ..." error message happens
+    for several different issues. So I list here several potential issues
     and their solutions if they exist.
+    R1 deals about too long lines in messages on Windows.
+    R2 deals about too long lines in messages on Unix.
+    R3 deals with quota reached.
 
 R1. On Windows, add --regexmess "s,(.{9900}),$1\r\n,g"
 
@@ -77,5 +133,8 @@ OPTIONS
        contains eight-bit characters to quoted-printable encoding.
 
        
-R2. 
+R3. Look for the quota limit on host2. If it is the current mailbox size, don't
+    search elsewhere. Increase the quota on the destination account.
 
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Admin_Authentication.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Admin_Authentication.txt,v 1.10 2019/10/31 23:13:50 gilles Exp gilles $
+$Id: FAQ.Admin_Authentication.txt,v 1.12 2021/02/19 13:44:18 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -19,16 +19,16 @@ server and its configuration.
 
 You have to follow this road:
  * Determine what is the imap software server
- * Find out wether it supports an admin account or not
+ * Find out whether it supports an admin account or not
  * If no, leave this reading, take an handkerchief and cry
  * If yes, search how the imap software server supports 
    an admin account (see below)
- * Figure out wether an admin account is actually configured or not,
+ * Figure out whether an admin account is actually configured or not,
- * If not then configure it,
+ * If not then configure it. If you can't configure it: handkerchief.
  * Finally, check an imapsync authentication via this admin account for
-   a standard user, option --justlogin is nice for doing this.
+   a standard user, the option --justlogin is nice for doing this.
  * Use it for all users.
- * Next, read their email messages, enter their personnal life etc. 
+ * Next, read their email messages, enter their personal life etc.
    I'm kidding.
 
 Good luck!
@@ -41,8 +41,8 @@ Known imap software servers supporting imap admin authentication:
   * Dovecot.    See the file FAQ.Dovecot.txt
   * Zimbra.     See the file FAQ.Zimbra.txt
   * Kerio       See the file FAQ.Kerio.txt
+  * Cyrus-imap. See the file FAQ.Cyrus.txt
   * UW-imap.    See the file FAQ.General.txt
-  * Cyrus-imap. See the file FAQ.General.txt
   * Sun Java Enterprise System/SunOne/iPlanet. See the file FAQ.General.txt
 
 Known imap software servers NOT supporting imap admin authentication:

FAQ.d/FAQ.Archiving.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Archiving.txt,v 1.18 2019/10/23 17:56:28 gilles Exp gilles $
+$Id: FAQ.Archiving.txt,v 1.24 2021/04/18 09:11:32 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -30,6 +30,9 @@ Q. How to archive a whole year in a subfolder?
 
 Q. How to archive a whole year in subfolders but without subsubfolders?
 
+
+Now the questions again with their answers.
+
 https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
 =======================================================================
 Q. How can I archive different accounts on the same destination account,
@@ -83,6 +86,10 @@ Examples:
   imapsync ... --user2 Bar --subfolder1 BarArchiv
 
 
+Read the previous FAQ item "How can I archive different accounts on the same 
+destination account, each account on a separate folder?"
+
+
 =======================================================================
 Q. How to move emails from one IMAP folder to another either on the 
    same IMAP server or a different one?
@@ -121,23 +128,37 @@ R2. Imapsync can't but rick-sanders-imap-tools can!
     is more dangerous than doing no backups at all. No backups makes
     people act in a safer way, well, usually...
     
-R3. Use ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos. 
+R3. Use isync or mbsync
+    https://isync.sourceforge.io/
+
+R4. Use ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos. 
     ImportExportTools does not do IMAP as a destination.
 
+R5. There is a free online service (French language) allowing you to do
+    a Maildir or EML or HTML backup at 
+    https://www.powermail.fr/sauvegarder-une-boite-mail/
+    Prefer Maildir or EML format if you want to be able to restore the 
+    archive. HTML format will be ok to only view the messages, not 
+    restore them to so imap server account.
+
+I have never used any of this tools so asking me for support with them 
+won't go far away.
+
 =======================================================================
 Q. How to archive a whole year in a subfolder?
 
-R.
+R. Example:
 
   imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
-      --folder2 ARCHIVE_2018 --justfolders --dry
+      --folder2 ARCHIVE_2018 
 
 =======================================================================
 Q. How to archive a whole year in subfolders but without subsubfolders?
 
+R. Example:
 
   imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
-             --sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}' --justfolders --dry
+             --sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}' 
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Authentication_failure.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Authentication_failure.txt,v 1.9 2019/01/11 13:13:08 gilles Exp gilles $
+$Id: FAQ.Authentication_failure.txt,v 1.13 2021/03/02 19:24:22 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -17,6 +17,8 @@ Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2
 Q. Imapsync fails with the following error, what can I do?
    Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom 
 
+Now the questions again with their answers.
+
 
 =======================================================================
 Q. Imapsync fails with the following error, what can I do?
@@ -55,9 +57,11 @@ Here are some advices to get you pass this difficult stage of authentication:
 
 
    * It is sometimes very hard to quote correctly unusual characters,
-     especially on Windows. See 
+     especially on Windows (and Mac sometimes). See 
      https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
      https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
+     https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+     
      The quickest trick may be to change the password temporally
      with easy characters like the classical alphabet, a long
      string still ensure strong security.
@@ -69,20 +73,54 @@ Here are some advices to get you pass this difficult stage of authentication:
        * Try --authmech1 CRAM-MD5 (or --authmech2 CRAM-MD5)
        * Try --authmech1 PLAIN    (or --authmech2 PLAIN)
 
-   * If you want to play manually the IMAP protocol on your server, 
+    * Manually test a login via ssl.
-     here is an example with the command telnet, user test1
+      Use either ncat or telnet-ssl or openssl commands like in the 
-     and password secret1 on the host test.lamiral.info.
+      following examples with imap.gmail.com server:
 
-     telnet test.lamiral.info 143
+  ncat --ssl -C imap.gmail.com 993
+  telnet-ssl -z ssl  imap.gmail.com 993
+  openssl s_client -crlf  -connect imap.gmail.com:993
+
+Replace imap.gmail.com by your imap server.
+The three commands do the same work here.
+
+Tipical dialog for an imap LOGIN command:
+  
+* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
+a LOGIN "gilles.lamiral@gmail.com" "secret" 
+* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
+a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
+b LOGOUT
+* BYE LOGOUT Requested
+b OK 73 good day (Success)
+
+The client part you have to type is 
+a LOGIN ... 
+b LOGOUT
+while replacing ... by your credentials values,
+other lines are the server responses. Letters 
+a and b are identifiers of IMAP commands, you have to 
+type them but you can chose any string instead of a or b.
+
+Some other examples:
+
+     ncat --ssl test.lamiral.info 993
      c1 LOGIN test1 "secret1"
      c2 LOGOUT
 
-     Same example using an ssl conneection:
+     openssl s_client -crlf  -connect test.lamiral.info:993
+     c1 LOGIN test1 "secret1" 
+     c2 LOGOUT
+
+     ncat --ssl -C outlook.office365.com 993
+     a LOGIN gilles.lamiral@outlook.com topsecret
+     b LOGOUT
 
      telnet-ssl -z ssl test.lamiral.info 993
      c1 LOGIN test1 "secret1" 
      c2 LOGOUT
 
+
 =======================================================================
 Q. Imapsync fails with the following error, what can I do?
    Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom 

FAQ.d/FAQ.Bandwidth.txt

@@ -0,0 +1,46 @@
+
+$Id: FAQ.Bandwidth.txt,v 1.2 2021/04/19 11:46:47 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Bandwidth.txt
+
+
+=======================================================================
+             Imapsync bandwidth used
+=======================================================================
+
+
+Questions answered in this FAQ are:
+
+Q. What is the bandwidth used by imapsync?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. What is the bandwidth used by imapsync?
+
+R. From the host where imapsync runs, it opens two imap connections, 
+   one with the source account at host1, one with the destination 
+   account at host2.
+   
+   So, the global bandwidth used is twice the volume of the source 
+   account, one volume to download the messages from host1, 
+   one volume to upload those messages to host2.
+   
+   If the host2 is already filled with the messages, imapsync doesn't
+   transfer them and then the volume transferred is small, this volume
+   is just the IMAP commands needed to identify the messages on both sides.
+   
+   There is no local cache of the email messages, except when the
+   message is very big, it is temporarily saved locally. 
+   
+   The biggest message seen so far on online /X is 3.08 GiB
+   while the biggest message transferred is 1.51 GiB so
+   I suspect a bug here. Drop me a note if you encounter the same
+   issue, I'll then dig into it, ie, I'll create a 2 or 3 GiB message
+   and play with it :-)
+
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Big_Mailbox.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Big_Mailbox.txt,v 1.4 2019/07/31 10:37:15 gilles Exp gilles $
+$Id: FAQ.Big_Mailbox.txt,v 1.7 2021/02/01 21:47:41 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -21,18 +21,20 @@ Q. How to deal with huge mailboxes, whose size is over dozens of GB?
 
 R1. It should be ok with imapsync.  
 
-In case imapsync seems to stall when sizing the folders or before
+Sometimes with big folders, imapsync seems to stall when sizing the
-syncing a huge folder and you wander wether it is doing something or
+folders or maybe before syncing a huge folder. So you wonder whether
-just frozen, you can add option --debugimap. Option --debugimap shows
+imapsync is doing something or being just frozen. A solution to
-what is currently done, it's quite a big output but it helps waiting,
+increase the verbosity is to add the option --debugimap. Option
-saying to ourselves "ok it's long but it's working, let's wait a
+--debugimap shows what is currently done, it's quite a big output but
-little more".  The real purpose of --debugimap is to show genuine IMAP
+it helps to wait, saying to ourselves "ok it's long but it's working,
-commands used and their responses. The problem with --debugimap is
+let's wait a little more".  The real purpose of --debugimap is to show
-that it is very verbose and so with a big mailbox it generates a very
+genuine IMAP commands used and their responses. The problem with
-BIG logfile.
+--debugimap is that it is very verbose and so with a big mailbox it
+generates a very BIG logfile.
 
-R2. If it is not ok with imapsync then consider spliting the whole 
+R2. If it is not ok with imapsync, the sync finally fails, then 
-sync into smaller chunks using --search
+consider spliting the whole sync into smaller chunks using 
+the option --search
 
 For example, select messages only for a given year:
 
@@ -41,5 +43,14 @@ For example, select messages only for a given year:
   imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018"
   imapsync ... --search "SENTSINCE 1-Jan-2019 SENTBEFORE 31-Dec-2019"
 
+R3. Use option --useuid
+Option --useuid speeds up the process but --useuid implies --usecache
+and --usecache may trigger some issues sometimes. Read:
+https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
+
+Don't use --useuid in Docker context, the cache will be destroyed 
+each time, or put it in a stable place, I don't know how to do it,
+I just know it's possible and not very difficult.
+
 =====================================================================
 =====================================================================

FAQ.d/FAQ.Contacts_Calendars.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Contacts_Calendars.txt,v 1.12 2019/04/30 13:04:18 gilles Exp gilles $
+$Id: FAQ.Contacts_Calendars.txt,v 1.14 2020/11/02 16:04:09 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -29,10 +29,10 @@ R. No, Imapsync can't migrate Contacts and Calendars.
   No way via IMAP, no way via imapsync.
   
   So it's a good idea to avoid syncing contacts and calendars when
-  using imapsync. See below
+  using imapsync. See at the end of this document
   "How can I avoid copying Calendar or Contacts folders?"
   
-  But see also the question at the end of this document
+  But see also the question below
   "How can I copy or synchronize Calendars or Contacts?".
 
 =======================================================================
@@ -41,6 +41,45 @@ Q. Can I copy or sync Chat messages with imapsync?
 R. No, Imapsync can't migrate Chat messages (those in GSuite for example).
    The explanation is the same as with Calendars or Contacts.
 
+=======================================================================
+Q. How can I copy or synchronize Calendars or Contacts?
+
+R1. Synchronizing Calendars or Contacts can't be done with imapsync. 
+    For detailed explanations, see the question above
+    "Can I copy or sync Calendars or Contacts with imapsync?".
+
+R2. Synchronizing Calendars or Contacts can be done, not
+    with imapsync, depending on the imap server software tool used.
+
+a) From Exchange to Exchange, export contacts and calendar to
+   PST format files on host1 and import them on host2.
+
+b) From Gmail to Gmail, export and import calendars in ical format, 
+   extension for those files is .ics.
+   Contacts can be copied using a csv file. See the help page
+   http://support.google.com/mail/bin/topic.py?hl=en&topic=1669027
+
+c) For Outlook, use caldavsynchronizer.
+   Homepage: http://caldavsynchronizer.org/
+   Outlook CalDav Synchronizer is a free Outlook Plugin, which synchronizes 
+   events, tasks and contacts between Outlook and Google, SOGo, Nextcloud or 
+   any other CalDAV or CardDAV server. 
+   Supported Outlook versions are Office 365, 2019, 2016, 2013, 2010 and 2007.
+   Free and Open Source software tool.
+
+
+d) Etc. Search the web. There's also specific tools and paid services.
+   There's no silver bullet to migrate Calendars and Contacts,
+   if you find one, tell me!
+   Also consider asking experts at Sumatra company 
+   https://twitter.com/sumatra_dev
+   http://sumatra.com/
+   
+e) I plan to make tools for that but so far nothing has began
+   more than this discussion:
+http://www.linux-france.org/prj/imapsync_list/msg01797.html
+http://www.linux-france.org/prj/imapsync_list/msg01811.html
+
 =======================================================================
 Q. How can I avoid copying Calendar or Contacts folders?
 
@@ -71,36 +110,5 @@ R. You can avoid synchronizing Calendar or Contacts folders with
     imapsync ... --exclude "^Calendar\$" --exclude "^Contacts\$" 
 
 
-=======================================================================
-Q. How can I copy or synchronize Calendars or Contacts?
-
-R1. Synchronizing Calendars or Contacts can't be done with imapsync. 
-    For detailed explanations, see the question above
-    "Can I copy or sync Calendars or Contacts with imapsync?".
-
-R2. Synchronizing Calendars or Contacts can be done, not
-    with imapsync, depending on the imap server software tool used.
-
-a) From Exchange to Exchange, export contacts and calendar to
-   PST format files on host1 and import them on host2.
-
-b) From Gmail to Gmail, export and import calendars in ical format, 
-   extension for those files is .ics.
-   Contacts can be copied using a csv file. See the help page
-   http://support.google.com/mail/bin/topic.py?hl=en&topic=1669027
-
-c) Etc. Search the web. There's also specific tools and paid services.
-   There's no silver bullet to migrate Calendars and Contacts,
-   if you find one, tell me!
-   Also consider asking experts at Sumatra company 
-   https://twitter.com/sumatra_dev
-   http://sumatra.com/
-   
-d) I plan to make tools for that but so far nothing has began
-   more than this discussion:
-http://www.linux-france.org/prj/imapsync_list/msg01797.html
-http://www.linux-france.org/prj/imapsync_list/msg01811.html
-
-
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Cyrus.txt

@@ -0,0 +1,127 @@
+#!/bin/cat
+$Id: FAQ.Cyrus.txt,v 1.2 2021/02/19 13:39:03 gilles Exp gilles $
+
+=======================================================================
+             Imapsync tips for the Cyrus imap server
+=======================================================================
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
+
+
+Questions answered in this FAQ are:
+
+Q. How to migrate from cyrus with an admin account?
+
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail but the tool copies everything in /home/user, how
+   can i avoid that?
+
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail directory.  When imapsync creates the folders in
+   the new cyrus imap server, it makes a folder "mail" and below that
+   folder puts all the mail folders the user have in /home/user/mail,
+   i would like to have all those folders directly under INBOX.
+
+Q. Migrating from Groupwise to Cyrus
+
+Now the questions again with their answers.
+
+
+======================================================================
+Q. How to migrate from cyrus with an admin account?
+
+R. Use:
+ 
+ imapsync ... \
+   --authuser1 admin_user  ----password1 admin_user_password \
+   --user1 foo_user --ssl1
+
+Instead of --ssl1 the alternative --tls1 can be used.
+With --authuser1, the option --authmech1 PLAIN is set
+automatically, you don't have to add it.
+
+PLAIN authentication is the only way to go with --authuser1 for now.
+So don't use --authmech1 SOMETHING with --authuser1 admin_user,
+it will not work.
+Same behavior with the --authuser2 option.
+
+Do not forget the option --ssl1 or --tls1 since PLAIN auth is only
+supported with ssl encryption most of the time. But it can
+work without --ssl1 nor --tls1 if PLAIN is permitted in clear text
+transmissions (the normal mode).
+
+Add the AdminAccount to admins line in /etc/imapd.conf
+Give AdminAccount lrswipkxtecda to the Cyrus Imap account 
+being migrated from, "joe" here.
+
+
+Here is an example:
+ imapsync \
+	--host1 server1 \
+	--user1 joe \
+	--authuser1 AdminAccount \
+	--password1 AdminAccountPassword \
+	--ssl1 \
+	--host2 server2 \
+	--user2 joe \
+	--password2 joespassonserver2 \
+	--exclude "^user\."
+
+=======================================================================
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail but the tool copies everything in /home/user, how
+   can i avoid that?
+
+Two solutions:
+
+R. Use
+   imapsync ... --include '^mail'
+
+R. or (better)
+   imapsync ... --subscribed --subscribe
+
+=======================================================================
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail directory.  When imapsync creates the folders in
+   the new cyrus imap server, it makes a folder "mail" and below that
+   folder puts all the mail folders the user have in /home/user/mail,
+   i would like to have all those folders directly under INBOX.
+
+R. Use 
+   imapsync ... --regextrans2 's/^mail/INBOX/' --dry
+   look at the simulation and if all transformations seem
+   good then remove the --dry option.
+
+
+=======================================================================
+Q. Migrating from Groupwise to Cyrus
+
+R. By Jamie Neil:
+
+I eventually managed to get the mail to migrate without errors using the 
+following options:
+
+--maxage 3650
+  - some messages just don't seem to want to transfer and produce the 
+    perl errors I mentioned before. This prevents the errors, but the
+    bad messages don't transfer.
+
+Even though the mail migrated OK, there are a couple of gotchas with 
+Groupwise IMAP:
+
+1) Some of the GW folders are not real folders and are not available
+to IMAP, the main problem one being "Sent Items". I could find no way
+of coping the contents of these folders. The nearest I got was to
+create a "real" folder and copy/move the sent items into it, but
+imapsync still didn't see the messages (I think because there is
+something funny about the reported dates/sizes).
+
+It think this problem has been rectified in GW6.5.
+
+2) Version 6.0.1 of the Groupwise Internet Connector sucks. I was 
+getting server aborts when I pushed it a bit hard! I eventually had to
+upgrade to 6.0.4 which seems to be a lot more stable.
+
+=======================================================================

FAQ.d/FAQ.DBmail.txt

@@ -0,0 +1,22 @@
+#!/bin/cat
+$Id: FAQ.DBmail.txt,v 1.1 2020/11/04 08:43:05 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.DBmail.txt
+
+=======================================================================
+              Imapsync tips for DBmail accounts.
+=======================================================================
+
+
+=======================================================================
+Q. How to migrate from or to DBmail?
+
+R. In order to avoid duplicates, use:
+
+  imapsync ...  --useheader Message-Id
+
+More explanations and details at
+https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
+

FAQ.d/FAQ.Docker.txt

@@ -1,50 +1,86 @@
 #!/bin/cat
-# $Id: FAQ.Docker.txt,v 1.6 2019/12/11 15:39:14 gilles Exp gilles $
+# $Id: FAQ.Docker.txt,v 1.7 2020/01/08 05:02:55 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt
 
-=======================================================================
+===========================================================================
             Installing and using imapsync docker image
-=======================================================================
+===========================================================================
 
 Questions answered in this FAQ are:
 
-Q. How can I install and use the imapsync Docker image on my system?
+
+Q. How can I install and use the imapsync Docker image on my amd64 system?
+
+Q. How can I install and use the imapsync Docker image on my i386 system?
 
 Q. How can I install or update only the imapsync Docker image on my system?
 
 Q. Where is the imapsync Docker hub location?
 
+Q. How can I mask the passwords on the command line without mounting
+   inside the container?
+
 Q. Any tips for the Docker Mailcow distribution?
 
 
 Now the questions again with their answers.
 
-=======================================================================
+===========================================================================
-Q. How can I install and use the imapsync Docker image on my system?
+Q. How can I install and use the imapsync Docker image on my amd64 system?
 
 R. Install Docker on your system. Once Docker is installed on your system,
    all you have to do in order to run imapsync is the command line:
 
   docker run gilleslamiral/imapsync imapsync <usual imapsync arguments>
 
-=======================================================================
+===========================================================================
+Q. How can I install and use the imapsync Docker image on my i386 system?
+
+R. You can't for now. It will throw this error when running it:
+
+  docker run gilleslamiral/imapsync imapsync
+  standard_init_linux.go:211: exec user process caused "exec format error"
+
+I plan to make the image multi-plateform. Drop me a note if you are
+eager to have it.
+
+===========================================================================
 Q. How can I install or update only the imapsync Docker image on my system?
 
 R. To install or update the imapsync image, run:
 
   docker pull gilleslamiral/imapsync
 
-=======================================================================
+===========================================================================
 Q. Where is the imapsync Docker hub location?
 
 R. Here:
 
   https://hub.docker.com/r/gilleslamiral/imapsync/
 
-=======================================================================
+===========================================================================
+Q. How can I mask the passwords on the command line without mounting 
+   anything inside the container?
+
+R. Use a file defining the environment variables 
+   IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2
+
+  cat ./secret.txt
+IMAPSYNC_PASSWORD1=secret1
+IMAPSYNC_PASSWORD2=secret2
+
+Use that file like this for a run:
+
+  docker run --env-file ./secret.txt gilleslamiral/imapsync imapsync ...
+
+See:
+https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
+https://stackoverflow.com/a/30494145/491175
+
+===========================================================================
 Q. Any tips for the Docker Mailcow distribution?
 
 R. With the Mailcow distribution, imapsync is running inside a Docker 
@@ -61,5 +97,5 @@ See the original Mathilde's comment:
 https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077
 
 
-=======================================================================
+===========================================================================
-=======================================================================
+===========================================================================

FAQ.d/FAQ.Domino.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-# $Id: FAQ.Domino.txt,v 1.12 2019/10/31 23:13:31 gilles Exp gilles $
+# $Id: FAQ.Domino.txt,v 1.13 2020/09/06 20:33:46 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -103,8 +103,9 @@ For Domino 853FP6 on Linux, we used this command on Unix:
   --prefix2 ""  --sep2 "\/" --messageidnodomain \
   --regexmess 's{\A(.*?(?! ^$))^Date:(.*?)$}{$1Migratedbyus:$2\nx-MailDate:$2}gxms'
 
+
 =======================================================================
-Q. Whan I use --maxage imapsync sees no messages at all.
+Q. When I use --maxage option, imapsync sees no messages at all.
    What can I do?
 
 R. Use --noabletosearch
@@ -113,6 +114,17 @@ R. Use --noabletosearch
 
 Same thing with --minage
 
+=======================================================================
+Q. On Domino this errors occurs: 
+   "Login failure, cannot convert Mailbox: code 02:46"
+
+R. Follow
+   https://atnotes.de/index.php?topic=44888.0
+
+Use:
+
+   load convert -e mail\mailfilename.nsf
+
 =======================================================================
 Q. My usual email software client like Thunderbird sees more messages
    than imapsync. How can it be possible?

FAQ.d/FAQ.Dovecot.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Dovecot.txt,v 1.7 2019/09/25 09:25:52 gilles Exp gilles $
+$Id: FAQ.Dovecot.txt,v 1.12 2021/04/30 13:02:06 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -18,8 +18,17 @@ Q. Subfolders are not created by imapsync and the error is
 
 Q. How to migrate to Dovecot with an admin/MasterUser account?
 
+Q. I have a mailbox that "dovecot backup" gives errors on that
+   I can't get rid of: "UID inserted in the middle of mailbox".
+   What can I do?
+
+Q. Imapsync encounters timeouts and Dovecot log says
+   "mailpiler dovecot: imap(xxx): Error: Timeout (180s) while waiting for dotlock for transaction log file ..."
+   What can I do?
+
 Now the questions again with their answers.
 
+
 =======================================================================
 Q. How to migrate to Dovecot with an admin/MasterUser account?
 
@@ -27,8 +36,22 @@ R. Dovecot uses the same syntax as uw-imap
 
   imapsync ... --user2="loginuser*admin_user" --password2 "admin_user_password"
 
-To setup a Dovecot MasterUser see 
+To setup a Dovecot MasterUser follow the document
-http://wiki2.dovecot.org/Authentication/MasterUsers
+https://doc.dovecot.org/configuration_manual/authentication/master_users/
+
+=======================================================================
+Q. I have a mailbox that "dovecot backup" gives errors on that
+   I can't get rid of: "UID inserted in the middle of mailbox".
+   What can I do?
+
+Q. Solution, run:
+
+  doveadm -v deduplicate -m -u martin mailbox A/Posted
+
+it will get rid of the duplicates (I can also be run without "-m")
+and this also allows "dovecot backup" to work.
+
+Thanks to Martin Ward for this input!
 
 =======================================================================
 Q. Subfolders are not created by imapsync and the error is
@@ -47,6 +70,15 @@ https://superuser.com/questions/813362/sub-folders-in-dovecot
 https://wiki2.dovecot.org/MailLocation/Maildir
 
 
+=======================================================================
+Q. Imapsync encounters timeouts and Dovecot log says
+   "mailpiler dovecot: imap(xxx): Error: Timeout (180s) while waiting for dotlock for transaction log file ..."
+   What can I do?
+
+R. It looks like an NFS issue with Dovecot. 
+   Solve it with Dovecot support or use something else than NFS.
+   I'll be glad to share here any solution you find.
+   (Thanks to Mate Skoblar for reporting this issue)
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Duplicates.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Duplicates.txt,v 1.20 2019/05/17 10:10:30 gilles Exp gilles $
+$Id: FAQ.Duplicates.txt,v 1.23 2021/04/22 13:28:41 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -9,6 +9,38 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
             Imapsync tips about duplicated messages issues.
 =======================================================================
 
+
+Questions answered in this FAQ are:
+
+Q. How does imapsync identify messages and duplicates?
+
+Q. How can I know if imapsync will generate duplicates on a second run?
+
+Q: I found multiple copies, duplicates, when I run imapsync twice or
+   more. What is hell is happening?
+   
+Q. imapsync calculates 479 messages in a folder but only transfers 400 
+   messages. What is happening?
+
+Q. imapsync doesn't synchronize duplicates by default but I want to.
+   How can I synchronize duplicates?
+
+Q. How can I remove duplicates in a lone account?
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. How does imapsync identify messages and duplicates?
+
+R. Imapsync by default identify messages by their headers "Message-Id" 
+   and "Received". Usually, for a given message, "Message-Id" appears one
+   time while multiple "Received" headers are common.
+   
+   For imapsync, messages with the same "Message-Id" and "Received" headers 
+   are consider identical, ie, duplicates.
+
+
 =======================================================================
 Q. How can I know if imapsync will generate duplicates on a second run?
 
@@ -32,7 +64,8 @@ Messages transferred              : 0
 Messages skipped                  : 1555
 
 =======================================================================
-Q: Multiple copies, duplicates, when I run imapsync twice ore more. 
+Q: I found multiple copies, duplicates, when I run imapsync twice or
+   more. What is hell is happening?
 
 R0. First, some explanations to understand the issue. 
 Normally and by default, imapsync doesn't generate duplicates.
@@ -172,7 +205,7 @@ trick to keep the --useheader filtering behavior is to use
 
 =======================================================================
 Q. imapsync calculates 479 messages in a folder but only transfers 400 
-   messages. What's happen?
+   messages. What is happening?
 
 R1. Unless --useuid is used, imapsync considers a header part 
    of a message to identify a message on both sides.
@@ -190,6 +223,38 @@ R2. With option --useuid imapsync doesn't use headers to identify
     messages on both sides but it uses their imap uid identifier.
     In that case duplicates on host1 are also transferred on host2.
 
+=======================================================================
+Q. imapsync doesn't synchronize duplicates by default but I want to.
+   How can I synchronize duplicates?
+
+R. Use the option --useuid
+   If you have already synchronized two mailboxes without --useuid then
+   using it right away will generate duplicates on host2. To avoid that
+   behavior, you have to perform a first run with --usecache to build
+   the local UID cache. Then the next runs with --useuid 
+
+   There are potentially issues with --usecache. They can be solved.
+   Read the document FAQ.Use_cache.txt 
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
+
+   So to finalise how to synchronize duplicates:
+   
+   imapsync ... --tmpdir . --usecache 
+
+   imapsync ... --tmpdir . --useuid
+   imapsync ... --tmpdir . --useuid
+   ...
+
+   Here --tmpdir value is the dot "." meaning "current directory".
+   Surrounding it with double-quotes is optional.
+
+   If the two mailboxes haven't been already synchronized then the
+   first run with --usecache is useless.
+
+   I shall add an option --syncduplicates that syncs duplicates 
+   without any cache needed and make it the default but it's not 
+   coded yet.
+
 =======================================================================
 Q. How can I remove duplicates in a lone account?
 

FAQ.d/FAQ.Emptying.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Emptying.txt,v 1.8 2019/04/30 12:52:10 gilles Exp gilles $
+$Id: FAQ.Emptying.txt,v 1.11 2020/10/13 16:24:43 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,6 +10,15 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Emptying.txt
                Imapsync tip to empty an account.
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. How to delete all emails of all folders of an account with imapsync?
+Q. How to delete all folders of an account with imapsync?
+Q. How to delete some folders of an account with imapsync?
+Q. How to revert a sync that went to the wrong account?
+
+Now the questions again with their answers.
+
 =======================================================================
 Q. How to delete all emails of all folders of an account with imapsync?
 
@@ -20,7 +29,7 @@ Option --noexpungeaftereach is just to speed up the deletions.
 
 Example (real example):
 
-  ./imapsync \
+  imapsync \
     --host1 test.lamiral.info --user1 empty --password1 secret \
     --host2 test.lamiral.info --user2 empty --password2 secret \
     --delete1 --noexpungeaftereach
@@ -31,10 +40,55 @@ Q. How to delete all folders of an account with imapsync?
 R. Use the same account on both sides with the options 
     --delete1 --noexpungeaftereach --delete1emptyfolders
     
-  ./imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders
+  imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders
 
 It won't delete the folder INBOX since INBOX is mandatory in imap.
 
 
+=======================================================================
+Q. How to delete some folders of an account with imapsync?
+
+R. Use the same account on both sides with the options 
+    --delete1 --noexpungeaftereach --delete1emptyfolders
+    like when deleting all folders BUT select only the folders
+    you want to delete by using options to select folders,
+    like --folder or --folderrec or --include or --exclude 
+
+Example, to delete the folder Blabla
+
+  imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders --folder Blabla
+
+To delete the four folders Kalender Kontakte Aufgaben Notizen:
+
+  imapsync ... --include 'Kalender|Kontakte|Aufgaben|Notizen' \
+                 --delete1 --noexpungeaftereach --delete1emptyfolders
+
+I strongly recommend to use --dry first!
+
+To get an overview of how to select folders, see
+https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
+
+=======================================================================
+Q. How to revert a sync that went to the wrong account?
+
+R. If all the wrong emails have some common characteristic that is not
+ shared by the good one, then you can search for them and delete them
+ using the same account as source and destination. For example, if all
+ the messages have the same email destination address as
+ "john_smith@example.com", then you can remove them with the following
+ command:
+   
+   imapsync ... --search "TO john_smith@example.com" --delete1 
+
+ Be very careful with deletions! Since this section deals with something
+ that was made wrong, you're in good place to make a new one!
+ 
+ There is many possibilities to select messages with different criterium.
+ See some of them by reading the document 
+ https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
+
+ If the wrongly emails messages synced have no special common 
+ characteristic then take an handkerchief and cry.
+
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Exchange.txt

@@ -1,55 +1,46 @@
 
-$Id: FAQ.Exchange.txt,v 1.57 2019/12/04 18:26:28 gilles Exp gilles $
+$Id: FAQ.Exchange.txt,v 1.74 2021/03/23 14:53:33 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
 
 
+
 =======================================================================
-            Imapsync tips for Exchange 20xx and Office365.
+      Imapsync tips for Exchange 2000 2003 2007 2010 2013 2016 2019
 =======================================================================
 
 Questions answered in this FAQ are:
 
-Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
+Q. Can I use imapsync to transfer from or to Exchange accounts?
 
-Q. How to sync from XXX to Exchange 2010/2013/2016
+Q. How to sync from XXX to Exchange 2010/2013/2016/2019
-
-Q. How to sync from Office365 to XXX?
-
-Q. How to sync from XXX to Office365
-
-Q. For Office365 I have double and triple checked the username and 
-   password spelling but I still get a "LOGIN failed". Any clue?
 
 Q. I see "NO Maximum size of appendable message has been exceeded"
-   What can I do with that? Happens with Office365 or Exchange 2016.
+   What can I do with that?
 
-Q. Office365 throttles the sync and says: 
+Q. Exchange throttles the sync and says: 
    "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
    What can I do with that?
 
-Q. Office365 refuses to create the folder named "Files" with the error
+Q. For Exchange I have double and triple checked the username and
-   "NO Folder name is reserved". What happens?
+   password spelling but I still get a "LOGIN failed". Any clue?
-
-Q. Office365 users complain that a folder named "Files" contains 
-   messages with no sender.
 
 Q. Exchange fails with "User is authenticated but not connected".
 
 Q. Exchange fails with "BAD Command received in Invalid state".
 
-Q. From XXX to Exchange 2013 or Office365, read receipts are all
+Q. From XXX to Exchange 2013, read receipts are all resent again 
-   resent again after a sync. Even for old messages. How can I fix that?
+   after a sync. Even for old messages. How can I fix that?
 
 Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
    DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
 
-Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
+Q. From XXX to Exchange 2010/2013 I get this error message
    sometimes: "BAD Command Argument Error 11". What does it mean?
 
-Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does 
+Q. From XXX to Exchange 2010 or 2013, the flag Flagged does 
    not seem to be well synced. What can I do?
 
 Q. How to migrate from or to Exchange 2007/2010/2013 with an
@@ -57,10 +48,7 @@ Q. How to migrate from or to Exchange 2007/2010/2013 with an
 
 Q. How to migrate from or to Exchange 2016 with an admin/authuser account?
 
-Q. How to migrate from or to Office 365 with an admin/authuser account?
+Q. How to migrate from or to Exchange 2003 with an admin/authuser account?
-
-Q. How to migrate from or to Exchange 2003 with an admin/authuser 
-   account?
 
 Q. Couldn't create folder [trash] "Mailbox already exists".
 
@@ -78,20 +66,21 @@ Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server?
 Now the questions again with their answers.
 
 =======================================================================
-Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
+Q. Can I use imapsync to transfer from or to Exchange accounts?
 
-R. Yes. But IMAP access to a Exchange or Office365 account is not always 
+R. Yes. But IMAP access to an Exchange account is not always 
    allowed by default so it has to be allowed in the server configuration 
    part.
 
    Exchange:  https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help
-   Office365: https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
 
 =======================================================================
-Q. How to sync from XXX to Exchange 2010/2013/2016
+Q. How to sync from XXX to Exchange 2010/2013/2016/2019
 
 R0. IMAP is not enable by default on Exchange, see how to enable it:
 https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help
+Also read
+http://clintboessen.blogspot.com/2018/03/binding-certificate-breaks-imap-or-pop.html
 
 
 R1. Following is a command line resume that solves most encountered 
@@ -135,73 +124,66 @@ that can be used like:
 
 To get reformime, install the "maildrop" package.
 
-=======================================================================
+On Linux again, there is a good Python script in the tarball that can 
-Q. How to sync from Office365 to XXX?
+fix several things that Exchange or O365 have issues with.
 
-R. On Windows, use:
+Use it like this:
 
-  imapsync.exe ... --office1
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
 
-On Unix, use:
+It often does some miracles on messages.
-
-  imapsync ... --office1
-
-Option --office1 is like (release 1.970):
-  
-  imapsync ... --host1 outlook.office365.com \
-               --ssl1 \
-               --exclude "^Files$" 
 
 =======================================================================
-Q. How to sync from XXX to Office365
+Q. For Exchange I have double and triple checked the username and
-
-R0. IMAP is not enable by default on Office365, see how to enable it:
-https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
-https://docs.microsoft.com/en-us/exchange/allow-pop3-imap4-and-smtp-server-settings-to-be-viewed-by-end-users-in-outlook-web-app-exchange-2013-help
-https://support.context.io/hc/en-us/articles/213586243-Enabling-IMAP-for-Office365
-
-R. Here is a command line resume that solves most encountered issues when 
-   migrating to Office365. It's similar with Exchange except for some 
-   values. To understand or change the details you have to read 
-   next Q/R sections.
-
-
-  imapsync ... --office2
-
-which is equivalent to (in imapsync release 1.870):
-
-  imapsync ... \
-    --host2 outlook.office365.com \
-    --ssl2 \
-    --maxsize 45000000 \
-    --maxmessagespersecond 4 \
-    --disarmreadreceipts \
-    --regexmess "s,(.{10239}),$1\r\n,g" \
-    --f1f2 "Files=Files_renamed_by_imapsync"
-
-
-On Linux, you can also try the "reformime" command 
-that can be used like:
-
-  imapsync ... --maxlinelengthcmd "reformime -r7"
-
-To get reformime, install the "maildrop" package.
-
-
-=======================================================================
-Q. For Office365 I have double and triple checked the username and 
    password spelling but I still get a "LOGIN failed". Any clue?
 
-R1. Triple check the hostname then. Try all of these:
- * imap-mail.outlook.com
- * imap.outlook.com
- * outlook.office365.com
 
-R2. Also triple check a license is assigned to that account 
+R1. Disable double-step authentication, also known as 2-factor,
-    in Office365.
+    2-step authentication on the Azure/Active Directory portal.
+
+R2. Microsoft introduced something called "security defaults" which is
+enabled by default for new tenants. One of the rules blocks IMAP
+access as of imapsync.  The funny thing is that you can't disable
+single rules of this security package without buying additional
+licenses.  Switching the whole thing off allows the login.
+
+Also, disable double-step authentication on the Azure/Active Directory
+portal.
+
+See here:
+https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
+Thanks to Stephan Buhre for this R2 answer.
+
+R3. I want to migrate to Exchange 2019. I haven't been able to sync
+using admin credentials even if I've tried and made all the
+configuration mentioned in the FAQ.Exchange.
+The error was always "LOGIN failed".
+
+My problem was that I have been used an admin account which haven’t
+attached a mailbox in Exchange. This is my admin account which I use
+only for configuration in Exchange, not for emails.
+
+So, after I’ve created in Exchange a mailbox for this user, the sync
+started to work.  I can confirm that imapsync is working for Exchange
+2019, with plain authentication, and full permissions of admin account
+(this account must have mailbox attached) applied to regular user
+mailbox account.
+
+It doesn't work with the --authuser2 parameter. The syntax that
+works for me is:
+
+  imapsync ... --host2 192.168.xxx.yyy  --user2 domain.local\admin.account\regular.user --password2 "xxxxxxx"
+or 
+  imapsync ...--host2 192.168.xxx.yyy  --user2 admin.account@mycompany.com\regular.user --password2 "xxxxxxx" 
+
+Thanks to Ioan Nechita for this R3 answer.
+
+R4. Are there special characters in the password?
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
 
-R3. Try with a classic email client like Thunderbird and the same
-    parameters.   
 
 =======================================================================
 Q. Exchange fails with "User is authenticated but not connected".
@@ -221,32 +203,16 @@ R2. Miguel Alameda reported understanding and solving this issue
 
 =======================================================================
 Q. I see "NO Maximum size of appendable message has been exceeded" 
-   What can I do with that? Happens with Office365 or Exchange 2016
+   What can I do with that?
 
-R0. It happens with Office365 and Exchange 2016.
+R1.  With Exchange 2019
-    See R1 for Office365
-    See R2 for Exchange 2016
 
-R1. Office365 supports send/receive max message sizes of up to 150MB 
+Change the recipient Receive/Send size:
-   but you need to make changes in your tenant(s) to support it.
 
-The following PowerShell command will increase the message sizes that 
+  Set-mailbox -Identity user@contoso.com -MaxReceiveSize 100mb -MaxSendSize 100mb
-can be sent/received.  The trick in getting IMAPSync to work is to 
-apply these settings to the accounts performing the migration,
-NOT the accounts associated with the target mailbox (assuming you're
-using service accounts to perform transfers on behalf of users).
 
-  Set-mailbox -Identity $UPN -MaxReceiveSize 150mb -MaxSendSize 150mb
+Thanks to Pasternak Denis for this Exchange 2019 tip.
 
-e.g. 
-
-  Set-mailbox -Identity "migrationaccount@testtenant.onmicrosoft.com" -MaxReceiveSize 150mb -MaxSendSize 150mb
-
-We're transferring data between Office 365 tenants so we set these 
-values on the migration acounts in the source and target tenants.
-
-Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto 
-for this FAQ item.
 
 R2. With Exchange 2016
 
@@ -275,7 +241,8 @@ Please note that this change may be lost each time you install an
 cumulative update. After the change, restart the services exchange 
 transport role, exchange information store and exchange imap4 Backend.
 
-(Source: https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde)
+Source in Deutch: 
+https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde
 
 This change made our Exchange installation accept bigger e-mails 
 in order to remove the imapsync Exchange error message
@@ -283,29 +250,19 @@ in order to remove the imapsync Exchange error message
 
 
 =======================================================================
-Q. Office365 throttles the sync and says: 
+Q. Exchange throttles the sync and says: 
    "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
    What can I do with that?
 
-
+R0. Exchange have throttle mechanisms to limit any huge 
-R. Exchange and Office365 have throttle mechanisms to limit any huge 
    usage. Sometimes imapsync transfers are too stressful for servers.
    The message 
    "Request is throttled. Suggested Backoff Time: 299961 milliseconds"
    comes from the imap Office365 server, imapsync just reports it
    before being disconnected from it.
 
-   To solve the throttles issues from 0365, there are two solutions
-   at least:
 
-R1. Call Microsoft Office365 and ask them to remove the limits on your 
+R1. Play with options --maxbytespersecond or --maxmessagespersecond
-  mailboxes. That's not a joke, they do it for 90 days usually, 
-  sometimes only after you reach the second technician you call, 
-  the first one usually been not enough competent to understand 
-  what you're talking about (I would be glad to remove this bad
-  fact).
-
-R2. Play with options --maxbytespersecond or --maxmessagespersecond
     or --exitwhenover
 
   imapsync ... --maxbytespersecond 100_000
@@ -317,61 +274,17 @@ R2. Play with options --maxbytespersecond or --maxmessagespersecond
 I don't know the upper value that avoid the default throttling from 0365
 and I guess it changes over time.
 
-R3. For Exchange, in case throttle appears anyway, fix them with:
+R2. For Exchange, in case throttle appears anyway, fix them with:
   https://docs.microsoft.com/en-us/exchange/change-user-throttling-settings-for-specific-users-exchange-2013-help
   See also:
   http://www.linux-france.org/prj/imapsync_list/msg02072.html 
+  (Subject: Re: [imapsync] From Exchange to Dovecot. Speed
+   Date: Mon, 03 Nov 2014 00:14:59 +0100)
 
   Sometimes restarting the Exchange server is needed to take
   into account the change in the configuration.
 
 =======================================================================
-Q. Office365 refuses to create the folder named "Files" with the error
-   "NO Folder name is reserved". What happens?
-
-R. The folder Files is a standard folder in Office365. It should not
-   be synced in IMAP. See the next question.
-
-=======================================================================
-Q. Office365 users complain that a folder named "Files" contains 
-   messages with no sender.
-
-R0. To fix this, add --exclude Files
-
-  imapsync ... --exclude Files
-
-  If you use --office1 then imapsync will add this exclusion 
-  automatically like using  the option:
-  
-  imapsync ... --exclude "^Files$" 
-
-  If you use --office2 then imapsync will add a renaming of any
-  "Files" folder on host1, like using the option:
-  
-  imapsync ... --f1f2 "Files=Files_renamed_by_imapsync"
-
-  The host2 account ends up with a folder named 
-  "Files_renamed_by_imapsync", but no complaining.
-
-R1. This folder "Files" seems to be a standard folder in Exchange Online, 
-   but it is not. The folder contains all attachments in every email 
-   that is in the mailbox, but without any headers.
-   This causes some confusion for users as these appear in their 
-   search results as duplicate lines but without the sender details 
-   or even the message body.
-
-   This folder seems to be usually hidden so IMAP clients can’t see it, 
-   but for some reason sometimes it becomes visible.
-
-R2. It looks to be a common problem with Exchange Online. 
-    I’m not sure what causes the folder to appear.
-
-   More info here:
-https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
-https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
-
-  (This question/answer is taken quasi verbatim from Perttu Aaltonen)
-=======================================================================
 Q. Exchange fails with "BAD Command received in Invalid state".
 
 R1. This message might happen when authenticating without ssl nor tls,
@@ -386,8 +299,8 @@ R2. This message might also happen when authenticating with ssl and tls.
    servers allow that.
 
 =======================================================================
-Q. From XXX to Exchange 2013 or Office365, read receipts are all
+Q. From XXX to Exchange 2013, read receipts are all resent again 
-   resent again after a sync. Even for old messages. How can I fix that?
+   after a sync. Even for old messages. How can I fix that?
 
 R. imapsync can remove the header containing this read receipt request.
 
@@ -436,7 +349,7 @@ R3. See also the FAQ FAQ.SSL_errors.txt
     https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
 
 =======================================================================
-Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
+Q. From XXX to Exchange 2010/2013 I get this error message
    sometimes: "BAD Command Argument Error 11". What does it mean?
 
 R. This error message comes from Exchange IMAP server when it 
@@ -444,17 +357,14 @@ R. This error message comes from Exchange IMAP server when it
 
   * Some messages are bigger than the size limit. 10 MB by default
     on Exchange. It can be upped by configuration for Exchange.
-	Sometimes a reboot is needed to take configuration changes 
+    Sometimes a reboot is needed to take configuration changes 
-	into account.
+    into account.
     If you can't fix this limit on Exchange then use option
     --maxsize 10000000 for 10 MB, change it if needed) to tell
     imapsync to skip those messages.
-    This value is 45 MB by default for Office365
 
   imapsync ... --maxsize 10000000 # 10 MB for Exchange
 
-  imapsync ... --maxsize 45000000 # 45 MB for Office365
-  
 
   * Quota reached. The whole account is full.
     It can be upped by configuration.
@@ -467,10 +377,6 @@ R. This error message comes from Exchange IMAP server when it
 
   imapsync ... --maxlinelength 9900 
 
-    Office365 supports 10500 characters line length:
-
-  imapsync ... --maxlinelength 10500
-
   In case you prefer fixing messages with long lines the hard way,
   instead of skipping them with --maxlinelength 9900, just use:
 
@@ -486,7 +392,7 @@ Exchange, which means Exchange leaves the session and say goodbye,
 come back later. Rerun a sync then.
 
 On Linux, there is a good Python script in the tarball that can 
-fix several things that Exchange or O365 have issues with.
+fix several things that Exchange have issues with.
 
 Use it like this:
 
@@ -495,7 +401,7 @@ Use it like this:
 It often does some miracles on messages.
 
 =======================================================================
-Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does 
+Q. From XXX to Exchange 2010 or 2013, the flag Flagged does 
    not seem to be well synced. What can I do?
 
 R. Use the following trick. Run imapsync twice, one with --regexflag
@@ -522,10 +428,9 @@ Q. How to migrate from or to Exchange 2007/2010/2013 with an
    
 R. The tricks comes from Michele Marcionelli and Benjamin Priestman:
 
-This doesn't work:
+This may works:
   imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ...
-
+or
-This might works:
   imapsync ... --user2 "domain\admin2\user2" --password2 adminpassword2 ...
 or 
   imapsync ... --user2 "admin2@domain\user2" --password2 adminpassword2 ...
@@ -554,6 +459,9 @@ and b) about a mailNickname which can be different to the username.
 As it seems you need to use the mailNickname (e.g. alias) defined 
 for user2, not the username itself.
 https://techcommunity.microsoft.com/t5/exchange-team-blog/understanding-login-strings-with-pop3-imap/ba-p/610683
+https://web.archive.org/web/20191014050543/https://techcommunity.microsoft.com/t5/Exchange-Team-Blog/Understanding-login-strings-with-POP3-IMAP/ba-p/610683
+the second link has the same content, I give it in case the first disappears,
+thanks to archive.org!
 
 A little note from John Becker.
 After you pointed out that I had issues with authorization, I realized that
@@ -599,45 +507,17 @@ Hi guys. For a long time I could not start synchronization, I use admin rights
 and the --authuser2 switch when I migrated from Mdaemon to Exchange2016. 
 I always got the error:
 "Host2 failure: Error login on [my.exchange.com] with user [mydomainuser] auth [PLAIN]: 2 NO AUTHENTICATE failed."
-In order for ImapSync to login to the user’s mailbox using administrator rights,
+In order for ImapSync to login to the user's mailbox using administrator rights,
 you need to give for the administrator the permissions to the mailbox of this
 user with the command:
-“Add-MailboxPermission -Identity DomainUser -User DomainAdmin -AccessRights FullAccess -InheritanceType All -AutoMapping $false”
+Add-MailboxPermission -Identity DomainUser -User DomainAdmin -AccessRights FullAccess -InheritanceType All -AutoMapping $false
 using Exchange Management Shell. 
 Without this, synchronization will not work. 
 Please, add this comment to the FAQ.Exchange.txt
 (done!)
 
 ======================================================================
-Q. How to migrate from or to Office 365 with an admin/authuser account?
+Q. How to migrate from or to Exchange 2003 with an admin/authuser account?
-
-Note from Yago Torres Fernandez:
-(a working command using admin/authuser on host2 Office 365)
-
-  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com  ^
-               --password2 XXXX --ssl2  ^
-
-but previously in Office365 you must do something like that, using powershell:
-
-  Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all
-
-PLAIN authentication is the only way to go with --authuser1 for now.
-So don't use --authmech1 SOMETHING with --authuser1 admin_user,
-it will not work.
-Same behavior with the --authuser2 option.
-
-Note from Martin Paulucci:
-I had to remove the domain part for the user
-but not for the admin.  Example:
-
-  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated
-
-See also:
-http://www.linux-france.org/prj/imapsync_list/msg02203.html
-
-======================================================================
-Q. How to migrate from or to Exchange 2003 with an admin/authuser 
-   account?
 
 R. Thomas Edgar wrote the following
 
@@ -711,8 +591,9 @@ R1. Be sure to have at least Exchange 2010 SP2 Rollup 5
 https://tribalchicken.net/imap-sync-issues-with-exchange-2010/
 
 R2. See also the Outlook side (thanks to Martin Hochreiter for this solution)
-https://www.howto-outlook.com/faq/archivenotworking.htm
 by changing Archive to sent/receive date
+https://www.howto-outlook.com/faq/archivenotworking.htm
+https://web.archive.org/web/20190914063435/http://www.howto-outlook.com/faq/archivenotworking.htm
 
 It's often shorter to change one server than thousands clients
 so R1 might be easier to do.

FAQ.d/FAQ.FirstClass.txt

@@ -0,0 +1,68 @@
+#!/bin/cat
+$Id: FAQ.FirstClass.txt,v 1.1 2021/01/26 14:03:58 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.FirstClass.txt
+
+
+=======================================================================
+         Imapsync tips for the FirstClass IMAP server.
+=======================================================================
+
+
+
+======================================================================
+Q. From FirstClass to XXX
+   http://www.firstclass.com/
+
+R. Migrating from FirstClass is not easy because FirstClass, strangely, 
+   does not show all messages via IMAP. To make it show all messages,
+   a trick, painful to follow by hand, is moving emails 
+   out and back in, for each folder. May be it can be done by a script.
+
+  FirstClass releases prior to release 12 do not shows the "Sent" 
+  folder in IMAP but FirstClass release 12 shows it.
+  I advice you to upgrade to FirstClass release 12 before leaving it
+  with imapsync or another imap tool.
+
+Here is a command line used to migrate from FirtClass release 12:
+
+  imapsync ... \
+    --tmpdir /var/tmp --usecache \
+    --useheader Message-ID \
+    --idatefromheader \
+    --addheader \
+    --regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
+    --regextrans2 "s/[\^]/_/g"       \
+    --regextrans2 "s/['\"\\\\]/_/g"  \
+    --regextrans2 "s,&AC8-,-,g" \
+    --regextrans2 "s,&APg-,oe,g" 
+
+On Windows:
+  imapsync.exe ... ^
+    --automap ^
+    --usecache ^
+    --useheader Message-ID ^
+    --idatefromheader ^
+    --addheader ^
+    --regextrans2 "s,(/|^) +,$1,g" ^
+    --regextrans2 "s, +(/|$),$1,g" ^
+    --regextrans2 "s/[\^]/_/g" ^
+    --regextrans2 "s/['\\]/_/g" ^
+    --regextrans2 "s,^&AC8-,-,g"  ^
+    --regextrans2 "s,^&APg-,oe,g"
+
+
+Special thanks to Kristian Wind and Joey Alexander for helping me
+writing this FAQ item. 
+See also this worth reading discussion in a Zimbra forum:
+https://web.archive.org/web/20120627043731/http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
+
+Another advice with FirstClass as host1, do not use --delete1emptyfolders since
+it will destroy all subfolders of any empty folder. This is a FirstClass bug:
+when asked to list a non-empty folder it says it has no children while it does
+have ones.
+See the details and explanations at
+https://github.com/imapsync/imapsync/issues/207
+

FAQ.d/FAQ.Flags.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Flags.txt,v 1.21 2019/03/08 19:04:32 gilles Exp gilles $
+$Id: FAQ.Flags.txt,v 1.26 2021/04/25 20:47:05 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -41,6 +41,8 @@ Q. Is it possible to sync labels and stars made by Thunderbird to
    Exchange categories? Or a way in Outlook to show labels created by
    Thunderbird?
 
+Now the questions again with their answers.
+
 =======================================================================
 Q. How to debug flag issues?
 
@@ -205,7 +207,7 @@ On Unix:
 
   imapsync ... --regexflag 's/\\FORWARDED//gi' 
 
-or
+or (on Unix)
 
   imapsync ... --regexflag "s/\\\\FORWARDED//gi" 
 
@@ -217,11 +219,18 @@ Unix:      imapsync ...     --regexflag 's/\\Indexed//gi'
 Windows:   imapsync.exe ... --regexflag "s/\\ATTACHED//gi" 
 Unix:      imapsync ...     --regexflag 's/\\ATTACHED//gi' 
 
-All usually problematic flags in one line:
+Removing usually problematic flags in one line:
 
-Windows:   imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED//gi" 
+Windows:   imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi" 
-Unix:      imapsync ...     --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED//gi' 
+Unix:      imapsync     ... --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi' 
 
+Removing all problematic flags I've seen so far in one line:
+Windows:   imapsync.exe ... --regexflag "s/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g"
+Unix:      imapsync     ... --regexflag 's/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g'
+
+Removing all flags but the standard ones from the IMAP RFC:
+Windows:   imapsync.exe ... --regexflag "s/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg"
+Unix:      imapsync     ... --regexflag 's/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg'
 
 =======================================================================
 Q. How to convert flags with $ to \ character?

FAQ.d/FAQ.Folders_Mapping.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Folders_Mapping.txt,v 1.20 2019/07/27 20:04:57 gilles Exp gilles $
+$Id: FAQ.Folders_Mapping.txt,v 1.23 2021/04/28 23:42:04 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,25 +11,93 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
              Imapsync tips about changing folders names.
 =======================================================================
 
-By default, imapsync syncs all folders, one by one, in alphanumeric order.
+Questions answered in this FAQ are:
 
-By default, folders names from host1 are reproduced identical on host2,
+Q. What is the imapsync default behaviour with folders?
-except for the prefix and the separator which are automatically adapted
-for host2.
 
-The IMAP protocol has a specific way to code folders names,
+Q. What encoding imapsync uses to deal with folder names?
+
+Q. How can I encode folder names for imapsync without getting a headache?
+
+Q. What are the easy ways to change folder names?
+
+Q. What should I know before playing with the --regextrans2 option?
+
+Q. How to remove the string "INBOX." in the name of destination folders?
+
+Q. How to change only INBOX name to Inbox_Migrated?
+
+Q. How to sync all folders to a single INBOX folder?
+
+Q. How to sync all folders to a single WHOLE folder?
+
+Q. How to sync a complete account in a subfolder called FOO?
+
+Q. How to substitute all characters dot "." by underscores "_" ?
+
+Q. How to substitute all doublequotes " by underscores _ ?
+
+Q. How to substitute all characters * % . and # by underscores _ ?
+
+Q. How to change folder names like this:
+[mail/Sent Items] -> [Sent]
+[mail/Test]       -> [INBOX/Test]
+[mail/Test2]      -> [INBOX/Test2]
+
+Q. Is it possible to synchronize all messages from one server to 
+   another without recreating the folder structure and the target server.
+
+Q. I would like to move emails from InBox to a sub-folder called,
+   say "2010-INBOX" based on the date. Like all emails received in the
+   year 2010 should be moved to the folder called "2010-INBOX".
+
+Q. How to sync a complete account in a subfolder called FOO,
+   the hard way?
+
+Q. How to transform all folder names to lowercase?
+
+Q. How to transform all folder names to uppercase?
+
+Q. How to transform all folder names to lowercase, except the first 
+   character in uppercase?
+
+
+Now the questions again with their answers.
+
+
+=======================================================================
+Q. What is the imapsync default behaviour with folders?
+
+R. Imapsync syncs all folders, one by one, in alphanumeric order,
+   reproducing their name identical from the source account at host1 to 
+   the destination account at host2, except for the prefix and the 
+   separator which are automatically adapted for host2.
+
+=======================================================================
+Q. What encoding imapsync uses to deal with folder names?
+
+
+R. The IMAP protocol has a specific way to code folders names,
 especially when these names use non-ascii 7bit characters.
 This encoding is called utf7imap. 
 
-Imapsync uses the same encoding as IMAP: utf7imap.
+Imapsync uses the same encoding as IMAP, it uses utf7imap.
 
-In order to well specify folders names on the command line, imapsync
+=======================================================================
-prints the complete folder list of both sides at the beginning of each run.
+Q. How can I encode folder names for imapsync without getting a headache?
+
+
+R. In order to well specify folders names on the command line, imapsync
+   prints the complete folder list of both sides at the beginning 
+   of each run.
 
 The left column is the encoding you have to use, without the first enclosing 
 square brackets [], the right column is the human utf8 view.
 
-This document gives most examples with the powerful but complex
+=======================================================================
+Q. What are the easy ways to change folder names?
+
+R. This document gives most examples with the powerful but complex
 option --regextrans2. Before using --regextrans2 you should 
 consider using --automap and --f1f2 because they are simpler 
 to understand and use.
@@ -47,8 +115,10 @@ to understand and use.
                        Add --debug to see what's really going on.)
 
 
+=======================================================================
+Q. What should I know before playing with the --regextrans2 option?
 
-Things to know and understand before playing with --regextrans2
+The things to know and understand before playing with --regextrans2 are:
 
 *) --regextrans2 is used to transform folders names.
 
@@ -101,39 +171,60 @@ Here
    ETA calculation and it's a supplementary check on folders.
 
 =======================================================================
-Q. Give examples about --regextrans2
+Q. How to remove the string "INBOX." in the name of destination folders?
 
 
-Examples: 
+R. To remove "INBOX." in the name of destination folders:
 
-1) To remove INBOX. in the name of destination folders
+On Linux:
 
   imapsync ... --regextrans2 's/^INBOX\.(.+)/$1/'
 
-2) To change only INBOX to Inbox_Migrated
+On Windows:
 
+  imapsync ... --regextrans2 "s/^INBOX\.(.+)/$1/"
+
+=======================================================================
+Q. How to change only INBOX name to Inbox_Migrated?
+
+R. 
    imapsync ... --regextrans2 's{^INBOX$}{Inbox_Migrated}'
 
 
-2a) To sync all folders to INBOX
+=======================================================================
+Q. How to sync all folders to a single INBOX folder?
+
+R. 
 
   imapsync ... --regextrans2 "s/.*/INBOX/"
 
+=======================================================================
+Q. How to sync all folders to a single WHOLE folder?
 
-2b) To sync a complete account in a subfolder called FOO
+R. 
 
-With imapsync release 1.641 and former, simply use
+  imapsync ... --regextrans2 "s,.*,WHOLE,"
+
+
+=======================================================================
+Q. How to sync a complete account in a subfolder called FOO?
+
+R. With imapsync release 1.641 and former, simply use:
 
   imapsync ... --subfolder2 FOO
 
-Examples to this subfolder problem for any release
+Examples to this subfolder problem for previous releases
 can be seen below, in another Q/R section.
 
 
-3) to substitute all characters dot "." by underscores "_"
+=======================================================================
+Q. How to substitute all characters dot "." by underscores "_" ?
+
+R.
     --regextrans2 "s,\.,_,g"
 
-3b) to substitute all doublequotes " by underscores _
+=======================================================================
+Q. How to substitute all doublequotes " by underscores _ ?
 
 On Linux/Unix:
 
@@ -143,7 +234,8 @@ On Windows:
 
   --regextrans2 s,\^",_,g
 
-3c) to substitute all characters * % . and # by underscores _
+=======================================================================
+Q. How to substitute all characters * % . and # by underscores _ ?
 
 On Linux/Unix:
 
@@ -154,13 +246,14 @@ On Windows:
   --regextrans2 "tr,*%.#,_,"
 
 You can increase the *%.# list by any unwanted character,
-plus, the order doesn't count. See 3d) for &
+plus, the order doesn't count. See below for &
 
-3d) It is a bad idea to substitute & characters since &
+It is a bad idea to substitute & characters since &
 is a character to encode non-ascii characters in IMAP folder names.
 
 
-4) to change folder names like this:
+=======================================================================
+Q. How to change folder names like this:
 [mail/Sent Items] -> [Sent]
 [mail/Test]       -> [INBOX/Test]
 [mail/Test2]      -> [INBOX/Test2]
@@ -170,12 +263,56 @@ On Linux/Unix:
     --regextrans2 's,^mail/Sent Items$,Sent,' \
     --regextrans2 's,^mail/,INBOX/,'
 
+a) I want folders 'Sent', 'Sent Messages', 'Gesendet' from host1 
+    to be merged to folder 'Sent' on host2.
+
+  --regextrans2 "s{Sent|Sent Messages|Gesendet}{Sent}"
+
+b) I do not want this rule a) to apply to any subfolders,
+    e.g. 'folder/Sent' or 'folder/Sent Messages' etc. but only
+    basefolders.
+
+So you have to add an anchor character ^ meaning "beginning of the string",
+and an anchor character $ meaning "end of the string" like this:
+
+  --regextrans2 's{^Sent$|^Sent Messages$|^Gesendet$}{Sent}'
+
+c) I want just a) to be combined with a subfolder, for example:
+
+  INBOX                -> Subfolder/INBOX
+  Sent                 -> Subfolder/Sent
+  Sent Messages        -> Subfolder/Sent
+  Gesendet             -> Subfolder/Sent
+  folder/Sent          -> Subfolder/folder/Sent
+  folder/Sent Messages -> Subfolder/folder/Sent
+  folder/Gesendet      -> Subfolder/folder/Sent
+
+To achieve this, use:
+
+  --subfolder2 Subfolder --regextrans2 's{Sent|Sent Messages|Gesendet}{Sent}'
+
+d) I want a) and b) to be combined with a subfolder, for example:
+
+  INBOX                -> Subfolder/INBOX
+  Sent                 -> Subfolder/Sent
+  Sent Messages        -> Subfolder/Sent
+  Gesendet             -> Subfolder/Sent
+  folder/Sent          -> Subfolder/folder/Sent
+  folder/Sent Messages -> Subfolder/folder/Send Messages
+  folder/Gesendet      -> Subfolder/folder/Gesendet
+  ...
+
+This one is trickier to achieve:
+
+  --subfolder2 Subfolder --regextrans2 's{^Subfolder/Sent$|^Subfolder/Sent Messages$|^Subfolder/Gesendet$}{Subfolder/Sent}'
+
 
 ======================================================================
 Q. Is it possible to synchronize all messages from one server to 
-another without recreating the folder structure and the target server.
+   another without recreating the folder structure and the target server.
 
-R. Yes.
+R. Yes. This question has already been answered but a little of 
+   redundancy can't hurt, can he?
 
 For example, to synchronize all messages in all folders on host1
 to folder INBOX only on host2:
@@ -225,9 +362,10 @@ b) Manually:
 3) Cut/paste in INBOX.2010-INBOX
 
 
+
 =======================================================================
 Q. How to sync a complete account in a subfolder called FOO,
-   the hard way.
+   the hard way?
 
 R0. See the easy above using --subfolder2
 
@@ -259,22 +397,93 @@ On Windows:
 
    --regextrans2 "s,(.*),FOO/$1,"
 
-   or:
+=======================================================================
+Q. How to transform all folder names to lowercase?
 
-  c) Any separator, any prefix solution, FOO is the subfolder:
+R. 
 
-  It is a complicated line because every case is taken into account.
+First, some documentation to understand how the regexes work.
-  Type it in one line (or with the \ at the end of first line on Unix shells.
+perldoc perlre says:
 
-On Linux/Unix:
+  \l          lowercase next char (think vi)
+  \u          uppercase next char (think vi)
+  \L          lowercase till \E (think vi)
+  \U          uppercase till \E (think vi)
+  \E          end case modification (think vi)
+  \Q          quote (disable) pattern metacharacters till \E
 
-   --regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \
+
-   --regextrans2 's,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,'
+On Linux:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 's,([^/]+),\L$1\E,g'
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 's,([^.]+),\L$1\E,g'
 
 On Windows:
 
-  --regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," ^
+If the separator on host2 is the character slash "/", use:
-  --regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,"
+
+  --regextrans2 "s,([^/]+),\L$1\E,g"
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 "s,([^.]+),\L$1\E,g"
+
+
+=======================================================================
+Q. How to transform all folder names to uppercase?
+
+R. 
+
+On Linux:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 's,([^/]+),\U$1\E,g'
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 's,([^.]+),\U$1\E,g'
+
+On Windows:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 "s,([^/]+),\U$1\E,g"
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 "s,([^.]+),\U$1\E,g"
+
+=======================================================================
+Q. How to transform all folder names to lowercase, except the first 
+   character in uppercase?
+
+R. 
+
+On Linux:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 's,([^/]+),\u\L$1\E,g'
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 's,([^.]+),\u\L$1\E,g'
+
+On Windows:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 "s,([^/]+),\u\L$1\E,g"
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 "s,([^.]+),\u\L$1\E,g"
 
 
 =======================================================================

FAQ.d/FAQ.Folders_Selection.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Folders_Selection.txt,v 1.8 2019/07/27 20:04:46 gilles Exp gilles $
+$Id: FAQ.Folders_Selection.txt,v 1.11 2020/12/31 11:41:46 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,6 +10,34 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
                Imapsync tips to select folders.
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. What is the default imapsync behaviour about folders?
+
+Q. What are the options to select folders?
+
+Q. Is it ok to use --subfolder1 and --subfolder2 to just select folders?
+
+Q. How can I sync only one folder?
+
+Q. What are --subscribe and --subscribed for, and how can they be used?
+
+Q. How to exclude a folder hierarchy like "public"?
+
+Q. How to exclude only the INBOX folder?
+
+Q. How to exclude folders matching SPAM no matter the case, 
+   aka, how to be case insensitive?
+
+Q. How to make option --folder "MyFolder" be recursive?
+
+Q. How to change folders names or move them elsewhere?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. What is the default imapsync behaviour about folders?
+
 By default, Imapsync syncs all folders, one by one, in alphanumeric order.
 
 The IMAP protocol has a specific way to code folders names,
@@ -25,6 +53,52 @@ The left column is the encoding you have to use, without the first enclosing
 square brackets [], the right column is the human utf8 view.
 
 
+=======================================================================
+Q. What are the options to select folders?
+
+R. From https://imapsync.lamiral.info/README
+
+  OPTIONS/folders
+
+     --folder       str  : Sync this folder.
+     --folder       str  : and this one, etc.
+     --folderrec    str  : Sync this folder recursively.
+     --folderrec    str  : and this one, etc.
+
+     --folderfirst  str  : Sync this folder first. Ex. --folderfirst "INBOX"
+     --folderfirst  str  : then this one, etc.
+     --folderlast   str  : Sync this folder last. --folderlast "[Gmail]/All Mail"
+     --folderlast   str  : then this one, etc.
+
+     --nomixfolders      : Do not merge folders when host1 is case-sensitive
+                           while host2 is not (like Exchange). Only the first
+                           similar folder is synced (example: with folders
+                           "Sent", "SENT" and "sent"
+                           on host1 only "Sent" will be synced to host2).
+
+     --skipemptyfolders  : Empty host1 folders are not created on host2.
+
+     --include      reg  : Sync folders matching this regular expression
+     --include      reg  : or this one, etc.
+                           If both --include --exclude options are used, then
+                           include is done before.
+     --exclude      reg  : Skips folders matching this regular expression
+                           Several folders to avoid:
+                            --exclude 'fold1|fold2|f3' skips fold1, fold2 and f3.
+     --exclude      reg  : or this one, etc.
+
+
+=======================================================================
+Q. Is it ok to use --subfolder1 and --subfolder2 to just select folders?
+
+R. No.
+
+  If you do:
+  
+    imapsync ... --subfolder1 Foo --subfolder2 Foo
+  
+  then it ends up with all messages from Foo going to Foo/INBOX
+
 
 =======================================================================
 Q. How can I sync only one folder?
@@ -56,23 +130,15 @@ R. In the IMAP protocol each user can subscribe to one or more folders.
 
 
 =======================================================================
-Q. I want to exclude a folder hierarchy like "public"
+Q. How to exclude a folder hierarchy like "public"?
 
 R. Use:
 
---exclude "^public\."
+  --exclude "public"
-or maybe
---exclude '^"public\.'
-
-In the example given the character "." is the folder separator, you
-can omit it. Just take the string as it appears on the imapsync
-output line :
-
-From folders list : [INBOX] [public.dreams] [etc.]
 
 
 =======================================================================
-Q. I want to exclude only INBOX
+Q. How to exclude only the INBOX folder?
 
 R. Use:
 
@@ -83,8 +149,8 @@ A good way to see what will be done is to first use:
   imapsync ...  --exclude "^INBOX$" --justfolders --nofoldersizes --dry
 
 =======================================================================
-Q. I want to exclude folders matching SPAM no matter the case, 
+Q. How to exclude folders matching SPAM no matter the case, 
-   aka how to be case insensitive
+   aka, how to be case insensitive?
 
 R. Use:
 
@@ -96,7 +162,7 @@ A good way to see what will be done is to first use:
 
 
 =======================================================================
-Q. I want the --folder "MyFolder" option be recursive.
+Q. How to make option --folder "MyFolder" be recursive?
 
 Two solutions:
 
@@ -108,5 +174,11 @@ R2. Use --include "^MyFolder"
    Then the folder "MyFolder" and all its subfolders will be handled
    and only them.
 
+=======================================================================
+Q. How to change folders names or move them elsewhere?
+
+R. Read the document FAQ.Folders_Mapping.txt also available at
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
+
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Folders_Sizes.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Folders_Sizes.txt,v 1.3 2019/07/25 15:57:04 gilles Exp gilles $
+$Id: FAQ.Folders_Sizes.txt,v 1.4 2021/01/26 13:05:21 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -13,13 +13,13 @@ Questions answered in this FAQ are:
 
 Q. How can I know the size of all folders before launching a sync?
 
-Q. The folders sizes and the number of messages don't match between
+Q. After a sync, the folders sizes and the number of messages don't 
-   the source mailbox and the destination mailbox.
+   match between the source mailbox on host1 and the destination 
-   Why and what can I do?
+   mailbox on host2.
+   Why? What can I do about it?
 
 Now the questions again with their answers.
 
-
 =======================================================================
 Q. How can I know the size of all folders before launching a sync?
 
@@ -27,14 +27,18 @@ R. Use --justfoldersizes option.
 
 With the --justfoldersizes option, imapsync evaluates and displays the 
 folders sizes on both sides and then exits without doing any transfer.
+It also presents the total size of each account, the number of messages,
+the biggest message size, the number of folders.
+
 Example:
 
   imapsync ... --justfoldersizes
 
 =======================================================================
-Q. The folders sizes and the number of messages don't match between
+Q. After a sync, the folders sizes and the number of messages don't 
-   the source mailbox on host1 and the destination mailbox on host2.
+   match between the source mailbox on host1 and the destination 
-   Why and what can I do?
+   mailbox on host2.
+   Why? What can I do about it?
 
 R. There are several reasons to explain the different counts between
    source and destination after a sync.
@@ -66,11 +70,13 @@ R. There are several reasons to explain the different counts between
 
 6) Cosmic rays...
 
-The important part to take into account is the last lines of imapsync
+The important part to take into account for a good and well done sync 
-if you encounter the line:
+is the last lines of imapsync. A good sync is when you encounter the
-"The sync looks good, all X identified messages in host1 are on host2."
+three final lines:
-and also no unidentified messages.
 
+"The sync looks good, all xxxx identified messages in host1 are on host2."
+"There is no unidentified message on host1."
+"Detected 0 errors"
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.General.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-# $Id: FAQ.General.txt,v 1.242 2019/12/11 20:28:33 gilles Exp gilles $
+# $Id: FAQ.General.txt,v 1.244 2021/02/19 13:38:39 gilles Exp gilles $
 
 =======================================================================
                      General FAQ for imapsync 
@@ -21,6 +21,8 @@ Q. What are the most important differences between the Unix shell syntax
 
 Q. How to install imapsync?
 
+Q. How to upgrade imapsync?
+
 Q. How to use imapsync?
 
 Q. Can you give some configuration examples?
@@ -77,17 +79,10 @@ Q. Is it possible to sync also the UIDs of the IMAP server?
 
 Q. The option --subscribe does not seem to work
 
-Q. With huge account (many messages) when it comes to reading the
-   destination server it comes out this error:
-   "To   Folder [INBOX.foobar] Not connected" 
-   What can I do?
-
 Q. Can Imapsync filter Spam during the sync?
 
 Q. How to migrate from uw-imap with an admin/authuser account?
 
-Q. How to migrate from cyrus with an admin account?
-
 Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
    Netscape servers with an admin account?
 
@@ -103,9 +98,6 @@ Q. I want to play with headers line and --regexmess but I want to leave
 Q. My imap server does not accept a message and warns
    "Invalid header". What is the problem?
 
-Q. The contact folder isn't well copied. 
-   How to copy the contact folder?
-
 Q: How can I write an .rpm with imapsync
 
 Q. Where I can read up on the various IMAP RFCs?
@@ -179,10 +171,16 @@ any \$1 by $1 (remove the \ before $).
 =======================================================================
 Q. How to install imapsync?
 
-R. Read the INSTALL files in the tarball. also available at
+R. Read the INSTALL files in the tarball. They are also available at
-   https://imapsync.lamiral.info/#doc
+   https://imapsync.lamiral.info/#install
+   and 
    https://imapsync.lamiral.info/INSTALL.d/
 
+=======================================================================
+Q. How to upgrade imapsync?
+
+R. Do like a fresh installation.
+
 =======================================================================
 Q. How to use imapsync?
 
@@ -416,6 +414,7 @@ R3 If you want to install the Perl module locally in a directory
 Q. How can I use imapsync with Mail::IMAPClient 2.2.9 perl module?
 
 R. Mail::IMAPClient 2.2.9 is no longer supported.
+   Help yourself.
 
 =======================================================================
 Q. How to verify imapsync.exe I got is the right file bit per bit?
@@ -512,8 +511,8 @@ R. imapsync doesn't talk POP3 but I think you mean UID in IMAP.
 Q. Is it possible to sync also the UIDs of the IMAP server?
 
 R. UIDs in IMAP are chosen and created by the server, not by the 
-   client software. imapsync is a client software. So UIDs cannot 
+   client software. imapsync is a client software command. 
-   be synced by any imap method.
+   So UIDs cannot be synced by any imap method.
 
    UIDs may be synced via a rsync command between the imap servers but
    it implies they are the same software, among other constraints.
@@ -526,16 +525,6 @@ R1. Use it with --subscribed
 R2. There is also the --subscribe_all option that subscribe
     to all folders on host2.
 
-=======================================================================
-Q. With huge account (many messages) when it comes to reading the
-   destination server it comes out this error:
-   "To   Folder [INBOX.foobar] Not connected" 
-   What can I do?
-
-R. May be spending too much time on the source server, the connection 
-   timed out on the destination server.
-   Try options --nofoldersizes 
-
 ======================================================================
 Q. Can Imapsync filter Spam during the sync?
 
@@ -560,46 +549,6 @@ R. Use the following syntax:
   imapsync ... --user1="loginuser*admin_user" --password1 "admin_user_password"
 
 
-======================================================================
-Q. How to migrate from cyrus with an admin account?
-
-R. Use:
- 
- imapsync ... \
-   --authuser1 admin_user  ----password1 admin_user_password \
-   --user1 foo_user --ssl1
-
-Instead of --ssl1 the alternative --tls1 can be used.
-With --authuser1, the option --authmech1 PLAIN is set
-automatically, you don't have to add it.
-
-PLAIN authentication is the only way to go with --authuser1 for now.
-So don't use --authmech1 SOMETHING with --authuser1 admin_user,
-it will not work.
-Same behavior with the --authuser2 option.
-
-Do not forget the option --ssl1 or --tls1 since PLAIN auth is only
-supported with ssl encryption most of the time. But it can
-work without --ssl1 nor --tls1 if PLAIN is permitted in clear text
-transmissions (the normal mode).
-
-Add the AdminAccount to admins line in /etc/imapd.conf
-Give AdminAccount lrswipkxtecda to the Cyrus Imap account 
-being migrated from, "joe" here.
-
-
-Here is an example:
- imapsync \
-	--host1 server1 \
-	--user1 joe \
-	--authuser1 AdminAccount \
-	--password1 AdminAccountPassword \
-	--ssl1 \
-	--host2 server2 \
-	--user2 joe \
-	--password2 joespassonserver2 \
-	--exclude "^user\."
-
 ======================================================================
 Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
    Netscape servers with an admin account?
@@ -717,22 +666,9 @@ Solutions c) and d) keep "From " lines information
 (normally it's useless to keep them)
 
 Best solutions are e) or d). 
-
+I prefer the d) solution because it fixes the issue
-=======================================================================
+and keeps the old "From" value while the e) solution removes 
-Q. The contact folder isn't well copied. 
+the old "From" value definitively.
-   How to copy the contact folder?
-
-R. Forget the destination server (choose the same)
-Change the script around line 1426
-       # ITSD
-       $new_id = $from->copy($t_fold,$f_msg);
-       #$new_id = $to->append_string($t_fold,$string, $flags_f, $d);
-
-and tried a copy of the mail instead an append_string. Because we are
-using the same server, we can use $from->copy Therefore we seem to not
-download and upload the message and therefore we do not have any
-format issues. And now it works fine. (Thanks to Hansjoerg.Maurer)
-
 
 ======================================================================
 Q: How can I write an .rpm with imapsync

FAQ.d/FAQ.Gmail.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Gmail.txt,v 1.65 2019/12/04 18:26:00 gilles Exp gilles $
+$Id: FAQ.Gmail.txt,v 1.79 2021/03/31 16:20:28 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,6 +11,7 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
 
 Questions answered in this FAQ are:
 
+
 Q. Can I use imapsync to transfer from or to Gmail accounts?
 
 Q. Even after several syncs the number of messages differs
@@ -19,6 +20,10 @@ Q. Even after several syncs the number of messages differs
 
 Q. How many days does it take to transfer X GB?
 
+Q. What is the Gmail free maximum storage size? 
+R. 15 GB in 2020. Need more?
+   https://gsuite.google.com/pricing.html
+
 Q. How to synchronize from Gmail to Gmail?
 
 Q. How to synchronize from XXX to Gmail?
@@ -26,12 +31,25 @@ Q. How to synchronize from XXX to Gmail?
 Q. How to synchronize from Gmail to XXX?
 
 Q. I can't authenticate with Gmail via IMAP
-   and Gmail says "Please log in via your web browser"
+   and Gmail says 
+   "Please log in via your web browser"
+   or it says 
+   "Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)"
 
 Q. Can not open imap connection on [imap.gmail.com]
    Unable to connect to imap.gmail.com
 
-Q. Can I transfer "Chat" messages with imapsync?
+Q. I want to sync the Sent folder only, what is it on Gmail?
+
+Q. The context is a Gmail account at the source, but not Gmail at the
+destination. I want cross-duplicates, different Gmail folders/labels
+generating duplicate messages across folders on the destination,
+except for the "All Mail" folder.
+
+Q. What are the Gmail imap names of the folders 
+   "Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"?
+
+Q. Can I transfer Gmail "Chat" messages with imapsync?
 
 Q. Can I safely use --useuid for Gmail transfers?
 
@@ -62,30 +80,70 @@ Q. Can I use imapsync to transfer from or to Gmail accounts?
 
 R. Yes. But IMAP access to a Gmail account is not allowed by default so 
 it has to be allowed in the Gmail configuration part:
- -> Settings
+ -> Do not go to "Manage your Google Account" (the upper right icon with your Google face)
- -> Forwarding and POP/IMAP
+ -> Instead, go to Gmail at https://mail.google.com/
- -> IMAP Access
+ -> Click on Settings, the gear button on the upper right of Gmail.
- -> Enable IMAP
+ -> Click on "See all settings" at the top of the pull down menu.
+ -> Go to the tab "Forwarding and POP/IMAP"
+ -> Go down to "IMAP Access"
+ -> Activate the radio button "Enable IMAP"
+ -> Click on the button "Save changes" at the bottom of the page.
 
-Also, there is the authentication issue to consider.
+That's not finished, there is also the authentication issue to consider.
 There are three different ways to authenticate to Gmail with Imapsync,
 either:
 
-* "Access for less secure apps"
+1) "Access for less secure apps"
-* "2-step verification"
+2) "2-step verification"
-* XOauth
+3) XOauth2 (not ready for individual Gmail authentication, only for groups)
 
-For the last one, XOauth, see the document 
+For the last one, XOauth2, see the document 
 https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
 
-1) "Access for less secure apps" is turned ON
+
+
+1) "Access for less secure apps", the first way to authenticate 
+   to Gmail with Imapsync. This is the easiest way because
+   you just have to turn one button on.
+
+Go to https://myaccount.google.com/
+Click on the Security tab on the left menu or go directly to
+https://myaccount.google.com/security?gar=1
 
  -> Verify 2-step verification is OFF
     https://myaccount.google.com/security?hl=en&pli=1&nlr=1#signin
+
+Near the end of the page,
  -> Turn ON "Access for less secure apps"
     https://www.google.com/settings/security/lesssecureapps
     https://support.google.com/accounts/answer/6010255?hl=en
 
+Here is what Google add on this "Access for less secure apps":
+"Some apps and devices use less secure sign-in technology, which makes your 
+account vulnerable. You can turn off access for these apps, which we recommend, 
+or turn it on if you want to use them despite the risks. 
+Google will automatically turn this setting OFF if it’s not being used."
+
+So after a while, this button will be turned off again.
+I suggest to turn it off once the job with imapsync is finished.
+
+If you forgot to also enable imap you end up with login failure like this one:
+"Host failure: Error login on [imap.gmail.com] with user [xxxx@gmail.com] auth [LOGIN]:
+NO [ALERT] Your account is not enabled for IMAP use. Please visit your Gmail 
+settings page and enable your account for IMAP access. (Failure)"
+
+After using imapsync on a Gmail account, sometimes you may encounter an email 
+with the title "Critical security alert for your linked Google Account"
+and the body 
+"Sign-in attempt was blocked for your linked Google Account
+xxxx@gmail.com
+Someone just used your password to try to sign in to your account from 
+a non-Google app. Google blocked them, but you should check what happened. 
+Review your account activity to make sure that no one else has access."
+Click on the button "Check Activity" and let Gmail know you were the author 
+of the sign in.
+
+
 Sometimes, with Gsuite, the login for the users are ok with imapsync
 for a while then they become forbidden with 
 "failure: Error login on [imap.gmail.com] with user [xxx] auth [LOGIN]: 
@@ -95,10 +153,28 @@ In that case, use:
 https://admin.google.com/AdminHome?hl=fr#ServiceSettings/notab=1&service=securitysetting&subtab=lesssecureappsaccess
 to allow "Access for less secure apps to all users"
 (Thanks to Sébastien R. for this input)
+Remark: even if the user has enabled less secure apps at Super Admin 
+ and reseller level, it still needs amended at 
+ https://myaccount.google.com/security
+ (Thanks Tony@gle for this last tip!)
 
-2) "2-step verification" is turned ON
 
-Follow
+2) "2-step verification" the second way to authenticate 
+   to Gmail with Imapsync.
+
+Go to
+https://myaccount.google.com/security
+and sign in with the account you want to transfer email to or from.
+Go to the "Signing in to Google" part of the security page and follow
+the process to activate the "2-step verification" until it is ok.
+
+Then, just below "2-step verification" click on the "App passwords"
+and follow the instruction to get a 16 characters password, on the
+pull down button "Select app", choose "Other (Custom name)" and call
+it imapsync for example.  Now use this this password to access the
+mailbox with imapsync.
+
+The following document details what I've just described:
 https://support.google.com/accounts/answer/185833?hl=en
 Even if Google says "Every App Password is only used once",
 it's false and the password can be used many times.
@@ -122,18 +198,25 @@ Q. How many days does it take to transfer X GB?
 R. Basically it takes X days to transfer X GB per account.
    Gmail has usage limits per day and use throttles when
    they are overtaken
-   http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
+   https://support.google.com/a/answer/1071518?hl=en
    From the previous link:
 
-   * it's 2X days to upload  X GB to Gmail, it's why I suggest to add
+   * Upload with IMAP: 500 MB per day.
-   --maxbytespersecond 10000 --maxbytesafter 1_000_000_000
+   It's 2X days to upload  X GB to Gmail, it's why I suggest to add
-   for uploading messages to Gmail
-   * it's X/2 days to download X BG from Gmail, it's why I suggest to add
    --maxbytespersecond 20000 --maxbytesafter 1_000_000_000
-   for downloading messages from Gmail
+   for uploading messages to Gmail
+   (should be --maxbytespersecond 5787 --maxbytesafter 500_000_000
+   but the higher values given work well)
    
-   That's theoretical values that always work in practice. Try
+   * Download with IMAP: 2500 MB per day.
-   upper values and see if they still work.
+   It's X/2.5 days to download X GB from Gmail, it's why I suggest to add
+   --maxbytespersecond 40_000 --maxbytesafter 3_000_000_000
+   for downloading messages from Gmail
+   (should be --maxbytespersecond 28_935 --maxbytesafter 2_500_000_000
+  
+   That's empirical values that work in practice.
+   Try upper values and see if they still work. 
+   I would be grateful if you report them to me.
 
    How Gmail says limits are reached?
    This is either a disconnection with
@@ -287,12 +370,11 @@ be 500_000_000, not 1_000_000_000, but 1GB is usually ok.
 --maxsleep 2 is to avoid sleeping more than 2 seconds when 
 the --maxbytespersecond value implies a long sleep between 2 copies.
 
---maxsize 25_000_000 is mandatory since Gmail limits messages size
+--maxsleep 0 cancels any imapsync bandwidth restriction given by
-up to 25 MB. This value increases over time, it was 10 MB some 
+the options --maxbytespersecond or --maxmessagespersecond
-years ago so you can try higher values. The Gmail page about
+It does not cancel the imap server bandwidth restrictions.
-this limit is https://support.google.com/mail/answer/6584
 
---automap is optional but it will save manual folder names
+--automap is optional but it will save manual folders names
 changes or the use of --regextrans2 or --f1f2 to map folder names.
 
 --expunge1 is optional. It deletes messages marked \Deleted on host1.
@@ -343,24 +425,25 @@ Q. How to synchronize from Gmail to XXX?
 R. Use this example:
 
 imapsync \
-           --user1 foo@gmail.com \
+           --user1     foo@gmail.com \
            --password1 gmailsecret \
-           --host2 localhost \
+           --host2     localhost \
-           --user2 tata \
+           --user2     tata \
+           --password2 tatasecret \
            --gmail1
 
 Or, replacing what does --gmail1 (values are from imapsync 1.970):
 
 imapsync \
-           --host1 imap.gmail.com \
+           --host1     imap.gmail.com \
            --ssl1 \
-           --user1 foo@gmail.com \
+           --user1     foo@gmail.com \
            --password1 gmailsecret \
-           --host2 localhost \
+           --host2     localhost \
-           --user2 tata \
+           --user2     tata \
            --password2 tatasecret \
            --maxbytespersecond 40_000 \
-           --maxbytesafter 2_500_000_000
+           --maxbytesafter 2_500_000_000 \
            --automap \
            --maxsleep 2 \
            --skipcrossduplicates \
@@ -384,48 +467,51 @@ allow you to use an upper value than 2 GBytes per 24h without
 disconnections.
 
 --useheader="X-Gmail-Received" --useheader "Message-Id" are not
-mandatory. I use them because I found (several years ago, it may
+mandatory. I use them because I found (several years ago, it may have
-have changed) that Gmail always adds a different header 
+changed) that Gmail always adds a different header "X-Gmail-Received:"
-"X-Gmail-Received:" to all messages it gets. So the identification
+to all messages it gets. So the identification by imapsync can not
-by imapsync can not fail using this header. "Message-Id" is there
+fail using this header. "Message-Id" is there for safety about this
-for safety about this Gmail rule.
+Gmail rule.
 
---automap is optional but it will save manual folder names
+--automap is optional but it will save manual folder names changes or
-changes or the use of --regextrans2 to map folder names.
+the use of --regextrans2 to map folder names.
 
 --regextrans2 "s,\[Gmail\].,,"
-If your destination imap server doesn't like "[Gmail]" name, 
+If your destination imap server doesn't like the "[Gmail]" name, 
 get rid of this "[Gmail]" part with that.
 
-You can select folders exported to imap within the gmail preferences,
+You can select folders exported to imap within the Gmail preferences,
-for example you may unselect all "System labels".
+for example, you may unselect all "System labels".
 
 --skipcrossduplicates is optional but it can save Gigabytes of hard
 disk memory. Within imap protocol, Gmail presents Gmail labels as
-folders, so a message labeled "Work" "ProjectX" "Urgent" ends up 
+folders, so a message labelled "Work" "ProjectX" "Urgent" ends up in
-in three different imap folders "Work" "ProjectX" and "Urgent"
+three different imap folders "Work" "ProjectX" and "Urgent" after an
-after an imap sync. Option --skipcrossduplicates prevent this behavior.
+imap sync. Option --skipcrossduplicates prevent this behavior.
 
-An issue with --skipcrossduplicates is that the first label synced
+An issue with --skipcrossduplicates is that the first label synced by
-by imapsync goes to its corresponding folder but other labels are
+imapsync goes to its corresponding folder but other labels are then
-then ignored. You can choose what labels have the 
+ignored. You can choose what labels have priority by using the
-priority by using the --folderfirst option. For example
+--folderfirst option. For example, --folderfirst "Work" will sync
---folderfirst "Work" will sync messages labeled "Work" before
+messages labelled "Work" before messages labelled "CanWait" or
-messages labeled "CanWait" or "Urgent". By default imapsync 
+"Urgent". By default imapsync syncs folders (Gmail labels) using the
-syncs folders (Gmail labels) using the classical alphanumeric order.
+classical alphanumeric order.
 
---folderlast "CanWait" will sync only messages that have the
+Option --folderlast "CanWait" will sync only messages that only have
-label CanWait and only it.
+the label CanWait.
 
---folderlast "[Gmail]/All Mail", in conjunction with option 
+Option --folderlast "[Gmail]/All Mail", in conjunction with option
---skipcrossduplicates, will only put in "[Gmail]/All Mail"
+--skipcrossduplicates will only put in "[Gmail]/All Mail" the messages
-the messages that are not labeled at all.
+that are not labelled at all.
 
 
 https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
 =======================================================================
 Q. I can't authenticate with Gmail via IMAP
-   and Gmail says "Please log in via your web browser"
+   and Gmail says 
+   "Please log in via your web browser"
+   or it says 
+   "Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)"
 
 R0. My current settings on two Gmail accounts are made
 with three steps 0) 1) 2)
@@ -441,6 +527,7 @@ https://support.google.com/accounts/answer/6010255?hl=en
 
 R1. See Coert Grobbelaar solution:
 https://web.archive.org/web/20150906230041/http://security.stackexchange.com/questions/86404/how-do-i-interact-with-google-to-import-email-via-imapsync
+https://support.google.com/accounts/answer/185833
 
 R2. I had the same issue one time (mars 2015) logging to Gmail 
 with imapsync.
@@ -450,6 +537,48 @@ it asked me to receive a code via a mobile, I said yes,
 I entered the code and everything went ok on next syncs
 with imapsync. 
 
+R1 again. Since people rarely follow links in a txt file, I reproduce
+Google documentation from the link
+https://support.google.com/accounts/answer/185833
+
+Sign in using App Passwords
+An App Password is a 16-digit passcode that gives a non-Google app or 
+device permission to access your Google Account. 
+App Passwords can only be used with accounts that 
+have 2-Step Verification turned on.
+
+When to use App Passwords
+To help keep your account secure, use "Sign in with Google" to 
+connect apps to your Google Account. 
+If the app you’re using doesn’t offer this option, you can either:
+
+Use App Passwords to connect to your Google Account
+Switch to a more secure app
+Create & use App Passwords
+Note: If you use 2-Step-Verification 
+( https://support.google.com/accounts/answer/185839 )
+and are seeing a "password incorrect" error when trying to access 
+your Google Account, an App Password may solve the problem.
+
+* Go to your Google Account https://myaccount.google.com/
+* On the left navigation panel, choose Security.
+* On the "Signing in to Google" panel, choose App Passwords. 
+  If you don’t see this option:
+  * 2-Step Verification is not set up for your account.
+  * 2-Step Verification is set up for security keys only.
+  * Your account is through work, school, or other organization.
+  * You’ve turned on Advanced Protection for your account.
+* At the bottom, choose Select app and choose the app you’re using.
+* Choose Select device and choose the device you’re using.
+* Choose Generate.
+* Follow the instructions to enter the App Password. 
+  The App Password is the 16-character code in the yellow bar on your device.
+* Choose Done.
+
+Most of the time, you’ll only have to enter an App Password once per 
+app or device, so don’t worry about memorizing it.
+
+
 
 =======================================================================
 Q. Can not open imap connection on [imap.gmail.com]: 
@@ -483,8 +612,83 @@ Or go to http://ping.eu/nslookup/ to get the resolution.
 
 Thanks to Chris Nolan to report, understand and fix this issue!
 
+
+
 =======================================================================
-Q. Can I transfer "Chat" messages with imapsync?
+Q. I want to sync the Sent folder only, what is it on Gmail?
+
+R1. Use:
+
+  imapsync ... --folder "[Gmail]/Sent Mail"
+
+R2. In fact, by configuration from the user, it can be another folder 
+    name presented in IMAP. Use --automap --dry to see this mapping.
+    See below 
+
+=======================================================================
+Q. The context is a Gmail account at the source, but not Gmail at the
+destination. I want cross-duplicates, different Gmail folders/labels
+generating duplicate messages across folders on the destination,
+except for the "All Mail" folder.
+
+R. This behavior, different Gmail labels generating duplicates across
+folders except for the "All Mail" folder, is obtained with a
+synchronization with no --gmail1 but instead --exclude "All Mail"
+followed by another synchronization with --gmail1 but without
+--exclude "All Mail".
+
+  imapsync ... --exclude "All Mail"
+  imapsync ... --gmail1
+
+This second sync is done to get the messages with no labels at all
+from the folder "All Mail", if they are some of them.
+
+Technically speaking, the first synchronization has not the option
+--skipcrossduplicates but the second one has the option
+--skipcrossduplicates and it also performs the Gmail "All Mail"
+folder/label at the end so only the messages that have no labels will
+end up in the "All Mail" destination folder (which is not Gmail,
+remember the context).
+
+Thanks to Mike Alexander for this suggestion. 
+Mike also provided a patch to implement that directly in imapsync.
+See https://github.com/imapsync/imapsync/pull/266
+(I refused it because well maintaining it will be very hard for me
+and I have other priorities).
+
+=======================================================================
+Q. What are the Gmail imap names of the folders 
+   "Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"?
+
+R. By default, Gmail imap names of the folders 
+   "Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"
+   are like this:
+
+Turned on automapping folders ( use --noautomap to turn off automapping )
+Host1: special [Gmail]/All Mail     = \All
+Host1: special [Gmail]/Drafts       = \Drafts
+Host1: special [Gmail]/Sent Mail    = \Sent
+Host1: special [Gmail]/Spam         = \Junk
+Host1: special [Gmail]/Starred      = \Flagged
+Host1: special [Gmail]/Trash        = \Trash
+
+Host2: special [Gmail]/All Mail     = \All
+Host2: special [Gmail]/Bin          = \Trash
+Host2: special [Gmail]/Drafts       = \Drafts
+Host2: special [Gmail]/Sent Mail    = \Sent
+Host2: special [Gmail]/Spam         = \Junk
+Host2: special [Gmail]/Starred      = \Flagged
+
+This is an excerpt from the imapsync output log done with the 
+--automap option. This listing is located just before the 
+folders listings.
+
+This mapping can be changed by configuration in the Gmail 
+web interface so the result may vary.
+
+
+=======================================================================
+Q. Can I transfer Gmail "Chat" messages with imapsync?
 
 R. No. 
    Even if you think you can because the Chat folder is seen in IMAP,

FAQ.d/FAQ.IceWarp.txt

@@ -0,0 +1,23 @@
+#!/bin/cat
+$Id: FAQ.IceWarp.txt,v 1.1 2020/02/25 10:49:07 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.IceWarp.txt
+
+=======================================================================
+              Imapsync tips for IceWarp accounts.
+=======================================================================
+
+
+=======================================================================
+Q. How to migrate from or to IceWarp?
+
+
+R. Use:
+
+  imapsync ...  --noabletosearch
+
+The option --noabletosearch is needed only if options --maxage or --minage
+are used. I don't know which --search commands could work, maybe none.
+

FAQ.d/FAQ.MailEnable.txt

@@ -0,0 +1,37 @@
+#!/bin/cat
+$Id: FAQ.MailEnable.txt,v 1.2 2020/07/07 10:45:56 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.MailEnable.txt
+
+
+=======================================================================
+            Imapsync tips for MailEnable imap server
+=======================================================================
+
+
+
+======================================================================
+Q: Syncing to MailEnable 10.31 and after
+R: No special imapsync option is required, default is ok.
+
+======================================================================
+Q: Syncing from MailEnable 1.75
+R: Use options:
+
+    imapsync ... --sep1 "/" --prefix1 ""
+
+======================================================================
+Q: Syncing from MailEnable 2.2
+R: Use options:
+
+    imapsync ... --sep1 "." --prefix1 ""
+
+======================================================================
+Q: Syncing to MailEnable until MailEnable release 10.31 not included
+R: Use options:
+
+    imapsync ... --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
+
+======================================================================

FAQ.d/FAQ.Massive.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Massive.txt,v 1.30 2019/07/24 12:01:41 gilles Exp gilles $
+$Id: FAQ.Massive.txt,v 1.31 2020/10/15 01:53:52 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -67,8 +67,26 @@ R2. To estimate the presync period, consider the mean imapsync transfer
     Double the time because the best scenario never happens.
     Triple the time because, well, real world is like that.
 
-R3. To estimate the last period, just rerun a complete presync,
+R3. An other way to better evaluate the end of the presync period can 
-    ie, resync all the mailboxes, the final sync should take 
+    be based on your actual data. Just apply a simple rule of three 
+    on the mailboxes already migrated to estimate the global end.
+    If it took X hours to finish Y% of the mailboxes, then it
+    will take 100*X/Y hours to finish 100% of the mailboxes.
+
+    Following the same idea but using matematical garbage, 
+    the ETA can be estimated like this: 
+    t_0      = time of global start (the start of the first presync)
+    t_now    = time of now.
+    Nb_total = total number of mailboxes to be migrated.
+    Nb_now   = number of mailboxes already migrated.
+    
+    then
+    
+    ETA = t_end = (t_now - t_0) * (Nb_total / Nb_now) + t_0
+
+
+R4. To estimate the last period, the final sync, just rerun a complete 
+    presync, ie, resync all the mailboxes, the final sync should take 
     the same amount of time.
     
 =======================================================================
@@ -178,7 +196,7 @@ Here comes some measurements.
      dstat -n -N eth0 6  # Linux only (in 2018)
 
    Another excellent tool to measure the network trafic is iftop.
-   The following command will monitor imap and imaps connections
+   The following command will monitor imap and imaps connexions
    on interfce eth0, only them, and sum them up:
    
      iftop -i eth0 -f 'port imap or port imaps' -B # Linux
@@ -250,9 +268,9 @@ Possible bottlenecks:
 
 - Bandwidth. 
   Usually available bandwidth is NOT a bottleneck.
-  Meanwhile, it can be a bottleneck on small Internet connections.
+  Meanwhile, it can be a bottleneck on small Internet connexions.
   Imapsync downloads messages from host1 and upload messages to host2,
-  consider this in case the connections are asymmetric.
+  consider this in case the connexion are asymmetric.
   
 - I/O on disks. 
   I/O are a classical bottleneck, almost always forgotten.

FAQ.d/FAQ.Messages_Selection.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Messages_Selection.txt,v 1.15 2019/07/24 12:02:11 gilles Exp gilles $
+$Id: FAQ.Messages_Selection.txt,v 1.17 2021/01/23 13:01:00 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -33,7 +33,53 @@ Q. What are the selection criteria available with --search option?
 =======================================================================
 Q. What messages imapsync syncs by default?
 
-R. By default, Imapsync syncs all messages, except duplicates.
+R1. By default, Imapsync syncs all messages, except duplicates.
+
+R2. You can alter the default all messages selection by using one or 
+    more of the following options:
+
+Extract from the README https://imapsync.lamiral.info/README
+
+  OPTIONS/message selection
+
+     --maxsize      int  : Skip messages larger  (or equal) than  int  bytes
+     --minsize      int  : Skip messages smaller (or equal) than  int  bytes
+
+     --maxage       int  : Skip messages older than  int days.
+                           final stats (skipped) don't count older messages
+                           see also --minage
+     --minage       int  : Skip messages newer than  int  days.
+                           final stats (skipped) don't count newer messages
+                           You can do (+ zone are the messages selected):
+                           past|----maxage+++++++++++++++>now
+                           past|+++++++++++++++minage---->now
+                           past|----maxage+++++minage---->now (intersection)
+                           past|++++minage-----maxage++++>now (union)
+
+     --search       str  : Selects only messages returned by this IMAP SEARCH
+                           command. Applied on both sides.
+                           For a complete set of what can be search see
+                           https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
+
+     --search1      str  : Same as --search but for selecting host1 messages only.
+     --search2      str  : Same as --search but for selecting host2 messages only.
+                           So --search CRIT equals --search1 CRIT --search2 CRIT
+
+     --maxlinelength int : skip messages with a line length longer than  int  bytes.
+                           RFC 2822 says it must be no more than 1000 bytes but
+                           real life servers and email clients do more.
+
+
+     --useheader    str  : Use this header to compare messages on both sides.
+                           Ex: Message-ID or Subject or Date.
+     --useheader    str    and this one, etc.
+
+     --usecache          : Use cache to speed up next syncs. Not set by default.
+     --nousecache        : Do not use cache. Caveat: --useuid --nousecache creates
+                           duplicates on multiple runs.
+     --useuid            : Use UIDs instead of headers as a criterion to recognize
+                           messages. Option --usecache is then implied unless
+                           --nousecache is used.
 
 =======================================================================
 Q. Is there a way we can specify a date range to sync emails? 
@@ -70,21 +116,27 @@ or
 Q. Is there a way we can specify an age to sync emails? 
    If yes, can you please share some examples?
 
+
 R. Yes, with the --maxage or the --minage option. 
 
-E.1 Sync only messages less than 2 days old:
+   Have in mind that the options --maxage and --minage can NOT be 
+   combined successfully with the --search command. If the --search 
+   option is present on the command line then --maxage and --minage 
+   are ignored.
+
+Example.1 Sync only messages less than 2 days old:
 
   imapsync ... --maxage 2
 
-E.2 Sync only messages more than 2 days old:
+Example.2 Sync only messages more than 2 days old:
 
   imapsync ... --minage 2
 
-E.3 Sync only messages more than 30 days old and less than 365 days old:
+Example.3 Sync only messages more than 30 days old and less than 365 days old:
 
   imapsync ... --minage 30 --maxage 365
 
-E.4 Sync only messages less than 30 days old or more than 365 days old:
+Example.4 Sync only messages less than 30 days old or more than 365 days old:
 
   imapsync ... --maxage 30 --minage 365
 
@@ -101,9 +153,9 @@ Full explanation:
                          past|----maxage+++++minage---->now (intersection)
                          past|++++minage-----maxage++++>now (union)
 
-C.1 By default,
+Caveat.1 By default,
 
-  option --maxage is implemented as a --search SENTSINCE 
+  Option --maxage is implemented as a --search SENTSINCE 
 RFC 3501 says: SENTSINCE <date>
 Messages whose [RFC-2822] Date: header (disregarding time and
 timezone) is within or later than the specified date.
@@ -151,12 +203,17 @@ Q. Can I migrate only mails with attachments?
 
 R. Use:
 
+  imapsync ... --search "HEADER Content-Disposition attachment" 
+
+or
+
   imapsync ... --search "HEADER Content-Type multipart/mixed" 
 
 or more generally:
 
   imapsync ... --search "OR HEADER Content-Disposition attachment HEADER Content-Type multipart/mixed" 
 
+The last one doesn't work on all imap servers, not on Gmail at least (tested 2020/06/16)
 
 =======================================================================
 Q. How can I move messages marked \Deleted from all folders to

FAQ.d/FAQ.Messages_Too_Big.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Messages_Too_Big.txt,v 1.3 2019/06/18 16:37:39 gilles Exp gilles $
+$Id: FAQ.Messages_Too_Big.txt,v 1.4 2020/10/21 09:21:48 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,6 +11,34 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt
 
 Questions answered in this FAQ are:
 
+Q. What does imapsync with big messages?
+
+Q. What can I do to transfer messages bigger than what allows the 
+   imap destination server?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. What does imapsync with big messages?
+
+R. By default, imapsync checks if the host2 imap server announces a 
+   message size limit with "APPENDLIMIT=xxxx" in its CAPABILITY response.
+   Then imapsync takes this value to set automatically the --maxsize
+   option, unless --maxsize is already set on the command line
+   and has a smaller value than the APPENDLIMIT value.
+   
+   If the host2 imap server doesn't announce any message size 
+   limit then imapsync syncs messages no matter their sizes.
+   
+   When a message is copied to the destination server, the destination 
+   server accepts it or not but it does it as a whole, it takes the 
+   full message or nothing. Nothing triggers an error in imapsync.
+
+   * Gmail     announces APPENDLIMIT=35651584 (October 2020).
+   * Office365 announces nothing.
+   * Dovecot   announces nothing.
+
+=======================================================================
 Q. What can I do to transfer messages bigger than what allows the 
    imap destination server?
 
@@ -22,7 +50,7 @@ R1. A solution is to truncate the message to the maximum size allowed
     on the software tool reading the email message.
 
 For example, 
-Gmail currently limits messages to 35651584 bytes (May 2019),
+Gmail currently limits messages to 35651584 bytes (October 2020),
 
 Starting with imapsync release 1.938 option --truncmess xxxx truncates
 messages bigger than the  given size xxxx.

FAQ.d/FAQ.Migration_Plan.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Migration_Plan.txt,v 1.6 2019/04/10 12:05:31 gilles Exp gilles $
+$Id: FAQ.Migration_Plan.txt,v 1.7 2021/03/02 22:31:36 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -41,12 +41,15 @@ Classical scenario, credentials triplets are different on both sides
    follow this scenario.
 
  * Presync all the mailboxes from the old server host1 to the new
-   server host2.  Presyncs can usefully be done with --delete2 in
+   server host2. If the imap server names are going to change their IP
-   order to get an exact sync. But never use the option --delete2 once
+   resolution then don't use those names, use names that will always
-   users have started to play with their new account on host2, their
+   match the same imap servers, or use their IP addresses.
-   play will be lost on the next presync, or when the MX is changed,
+   
-   since INBOX will start to receive new messages that are not on
+   Presyncs can usefully be done with --delete2 in order to get an
-   host1.
+   exact sync. But never use the option --delete2 once users have
+   started to play with their new account on host2, their play will be
+   lost on the next presync, or when the MX is changed, since INBOX
+   will start to receive new messages that are not on host1.
 
  * Decide a migration day/hour.
 

FAQ.d/FAQ.Office365.txt

@@ -0,0 +1,510 @@
+
+$Id: FAQ.Office365.txt,v 1.20 2021/03/17 20:42:35 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Office365.txt
+
+
+=======================================================================
+            Imapsync tips for Office365.
+=======================================================================
+
+Questions answered in this FAQ are:
+
+Q. Can I use imapsync to transfer from or to Office365 accounts?
+
+Q. Does IMAPSync support OAUTH2 authentication in Office 365?
+
+Q. How to sync from Office365 to XXX?
+
+Q. How to sync from XXX to Office365
+
+Q. For Office365 I have double and triple-checked the username and
+   password spelling but I still get a "LOGIN failed". Any clue?
+
+Q. I see "NO Maximum size of appendable message has been exceeded"
+   What can I do with that?
+
+Q. The imap connection to Office365 is not working very efficiently,
+   is there a solution to fix that?
+
+Q. Office365 throttles the sync and says: 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
+   What can I do with that?
+
+Q. The sync fails with many "Trying command when NOT connected!".
+   What can I do?
+
+Q. How to see or migrate public folders on Office365?
+R. https://docs.microsoft.com/en-us/exchange/collaboration/public-folders/migrate-to-exchange-online?view=exchserver-2019
+   https://www.exchangesavvy.com/moving-your-public-folders-to-office-365-what-you-need-to-know/
+
+Q. Office365 refuses to create the folder named "Files" with the error
+   "NO Folder name is reserved". What happens?
+
+Q. Office365 users complain that a folder named "Files" contains 
+   messages with no sender.
+
+Q. From XXX to Office365, read receipts are all resent again after sync.
+   Even for old messages. How can I fix that?
+
+Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
+   DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
+
+Q. From XXX Office365 I get this error message sometimes: 
+   "BAD Command Argument Error 11". What does it mean?
+
+Q. From XXX to Office365 the flag Flagged does not seem to be well synced. 
+   What can I do?
+
+Q. How to migrate from or to Office 365 with an admin/authuser account?
+
+Q. Couldn't create folder [trash] "Mailbox already exists".
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Can I use imapsync to transfer from or to Office365 accounts?
+
+R. Yes. But IMAP access to Office365 account is not always allowed by default 
+   so it has to be allowed in the server configuration part.
+   
+https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
+https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
+
+=======================================================================
+Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
+
+R. No. But I plan to add it during the year 2021.
+
+=======================================================================
+Q. How to sync from Office365 to XXX?
+
+R. On Windows, use:
+
+  imapsync.exe ... --office1
+
+On Unix, use:
+
+  imapsync ... --office1
+
+Option --office1 is like (release 1.970):
+  
+  imapsync ... --host1 outlook.office365.com \
+               --ssl1 \
+               --exclude "^Files$" 
+  
+=======================================================================
+Q. How to sync from XXX to Office365
+
+R0. IMAP is not enabled by default on Office365, see how to enable it:
+https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
+https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
+
+If IMAP cannot be enabled or if it doesn't work well, you can try DavMail 
+(http://davmail.sourceforge.net/) which acts as an IMAP gateway 
+through Outlook Web Access.
+
+R. Here is a command-line resume that solves most encountered issues when 
+   migrating to Office365.
+
+    imapsync ... --office2
+
+which is equivalent to (in imapsync release 1.870):
+
+  imapsync ... \
+    --host2 outlook.office365.com \
+    --ssl2 \
+    --maxsize 45000000 \
+    --maxmessagespersecond 4 \
+    --disarmreadreceipts \
+    --regexmess "s,(.{10239}),$1\r\n,g" \
+    --f1f2 "Files=Files_renamed_by_imapsync"
+
+
+On Linux, you can also try the "reformime" command 
+that can be used like:
+
+  imapsync ... --maxlinelengthcmd "reformime -r7"
+
+To get reformime, install the "maildrop" package.
+
+On Linux again, there is a good Python script in the tarball that can 
+fix several things that Exchange or O365 have issues with.
+
+Use it like this:
+
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
+
+It often does some miracles on messages.
+
+=======================================================================
+Q. For Office365 I have double and triple-checked the username and
+   password spelling but I still get a "LOGIN failed". Any clue?
+
+R0. Imapsync doesn't support OAUTH2 authentication for Office365 accounts yet.
+
+R1. Disable double-step authentication, also known as 2-factor,
+    2-step authentication on the Azure/Active Directory portal.
+
+R2. Also, check a license is assigned to that account in Office365.
+
+R3. Microsoft introduced something called "security defaults" which is enabled
+by default for new tenants. One of the rules blocks IMAP access as of imapsync. 
+The funny thing is that you can't disable single rules of this security 
+package without buying additional licenses.
+Switching the whole thing off allows the login.0
+Also, disable double-step authentication on the Azure/Active Directory portal.
+See here:
+https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
+Thanks to Stephan Buhre for this R3 answer.
+
+R4. Are there special characters in the password?
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+
+R5. Triple check the hostname then. Try all of these:
+ * outlook.office365.com
+ * imap-mail.outlook.com
+ * imap.outlook.com
+
+=======================================================================
+Q. I see "NO Maximum size of appendable message has been exceeded"
+   What can I do with that?
+
+
+R. Office365 supports send/receive max message sizes of up to 150MB 
+   but you need to make changes in your tenant(s) to support it.
+
+The following PowerShell command will increase the message sizes that 
+can be sent/received.  The trick in getting IMAPSync to work is to 
+apply these settings to the accounts performing the migration,
+NOT the accounts associated with the target mailbox (assuming you're
+using service accounts to perform transfers on behalf of users).
+
+  Set-mailbox -Identity $UPN -MaxReceiveSize 150mb -MaxSendSize 150mb
+
+e.g. 
+
+  Set-mailbox -Identity "migrationaccount@testtenant.onmicrosoft.com" -MaxReceiveSize 150mb -MaxSendSize 150mb
+
+We're transferring data between Office 365 tenants so we set these 
+values on the migration accounts in the source and target tenants.
+
+Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto 
+for this FAQ item.
+
+=======================================================================
+Q. The imap connection to Office365 is not working very efficiently,
+   is there a solution to fix that?
+
+R. Yes. Try DavMail
+   http://davmail.sourceforge.net/
+   I don't use it myself but a user, Yannick Palanque,  reported great results using it.
+
+=======================================================================
+Q. The sync fails with many "Trying command when NOT connected!".
+   What can I do?
+
+R. The --debugimap option can show you more details, especially some
+   message like:
+   "BAD Request is throttled. Suggested Backoff Time: 178755 milliseconds"
+   In that case, see the next faq item and its fixes.
+
+=======================================================================
+Q. Office365 throttles the sync and says: 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
+   What can I do with that?
+
+R. Office365 has throttle mechanisms to limit any huge usage.
+   Sometimes imapsync transfers are too stressful for servers.
+   The following message 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds"
+   comes from the imap Office365 server, imapsync just reports it
+   before being disconnected from it.
+
+   To solve the throttles issues from 0365, there are two solutions
+   at least:
+
+R1. Call Microsoft Office365 and ask them to remove the limits on your 
+  mailboxes. That's not a joke, they do it for 90 days usually, 
+  sometimes only after you reach the second technician you call, 
+  the first one usually be not enough competent to understand 
+  what you're talking about (I would be glad to remove this bad
+  fact).
+
+R2. Play with options --maxbytespersecond or --maxmessagespersecond
+    or --exitwhenover
+
+  imapsync ... --maxbytespersecond 100_000
+
+  imapsync ... --maxmessagespersecond 2
+
+  imapsync ... --exitwhenover 1_000_000_000
+
+I don't know the upper value that avoids the default throttling from 0365
+and I guess it changes over time.
+
+=======================================================================
+Q. Office365 refuses to create the folder named "Files" with the error
+   "NO Folder name is reserved". What happens?
+
+R. The folder Files is a standard folder in Office365.
+   It should not be synced in IMAP. See the next question.
+
+=======================================================================
+Q. Office365 users complain that a folder named "Files" contains 
+   messages with no sender.
+
+R0. To fix this, add --exclude Files
+
+  imapsync ... --exclude Files
+
+  If you use --office1 then imapsync will add this exclusion 
+  automatically like using  the option:
+  
+  imapsync ... --exclude "^Files$" 
+
+  If you use --office2 then imapsync will add a renaming of any
+  "Files" folder on host1, like using the option:
+  
+  imapsync ... --f1f2 "Files=Files_renamed_by_imapsync"
+
+  The host2 account ends up with a folder named 
+  "Files_renamed_by_imapsync", but no complaining.
+
+R1. This folder "Files" seems to be a standard folder in Exchange Online, 
+   but it is not. This folder contains all attachments in every email 
+   that is in the mailbox but without any headers.
+   This causes some confusion for users as these appear in their search
+   results as duplicate lines but without the sender details or even
+   the message body.
+
+   This folder seems to be usually hidden so IMAP clients can’t see it, 
+   but for some reason sometimes it becomes visible.
+
+R2. It looks to be a common problem with Exchange Online. 
+    I’m not sure what causes the folder to appear.
+
+   More info here:
+https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
+https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
+
+  (This question/answer is taken quasi verbatim from Perttu Aaltonen)
+
+=======================================================================
+Q. From XXX to Office365, read receipts are all resent again after sync.
+   Even for old messages. How can I fix that?
+
+R. Imapsync can remove the header containing this read-receipt request.
+
+On Unix or Windows use:
+
+  imapsync ... --disarmreadreceipts
+
+Since read receipts should be sent for unseen messages that will go 
+to a seen state after the migration, you could be strict and apply 
+the regex only to seen messages. 
+Selecting seen message can be done with:
+
+  imapsync ... --search1 "SEEN" --disarmreadreceipts
+
+A second run has to be run without the --disarmreadreceipts
+for unseen messages:
+
+  imapsync ... --search1 "UNSEEN" 
+
+If fact --disarmreadreceipts is just an option equivalent to:
+
+  --regexmess 's{\A(.*?(?! ^$))^Disposition-Notification-To:(.*?)$}{$1X-Disposition-Notification-To:$2}igxms' 
+
+That regex changes the header Disposition-Notification-To. It prefixes
+it with an X- so that it becomes inactive.
+Disposition-Notification-To: blabla
+becomes
+X-Disposition-Notification-To: blabla
+
+Thanks to David Karnowski for pointing and solving this issue.
+
+=======================================================================
+Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
+   DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
+
+R1. "SSL read or write error" happens sometimes, it isn't related to 
+    imapsync directly but to the ssl underlying library when communicating
+    with Exchange in TLS/SSL encrypted mode. 
+    Next runs should put the sync further, so rerun the syncs 
+    until it is well completed.
+
+R2. Another solution is to remove --tls or --ssl options for Exchange
+    and accept clear text syncs.
+
+R3. See also the FAQ FAQ.SSL_errors.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
+
+=======================================================================
+Q. From XXX Office365 I get this error message sometimes: 
+   "BAD Command Argument Error 11". What does it mean?
+
+R. This error message comes from the Office365 IMAP server when it 
+   encounters any problem. Most of the time it is one of the following:
+
+  * Some messages are bigger than the size limit. 
+    45 MB by default on Office365.
+    I don't know if it can be upped by configuration for Office365.
+    If you can't fix this limit on Office365 then use the option
+    --maxsize 45000000 for 45 MB to tell imapsync to skip those messages.
+
+  imapsync ... --maxsize 45000000 # 45 MB for Office365
+  
+
+  * Quota reached. The whole account is full.
+    It can be upped by configuration.
+
+  * You use --synclabels --resynclabels from a previous command line 
+    related to syncing from Gmail to Gmail. Remove them.
+
+  * Some messages have some lines too long. Use option --maxlinelength
+    to skip messages whose max line length is over some bytes.
+    --maxlinelength 1000 is an RFC2822 must but most servers support
+    higher values. Exchange supports 9900 characters line length:
+
+  imapsync ... --maxlinelength 9900 
+
+    Office365 supports 10500 characters line length:
+
+  imapsync ... --maxlinelength 10500
+
+  In case you prefer fixing messages with long lines the hard way,
+  instead of skipping them with --maxlinelength 9900, just use:
+
+  On Windows
+  imapsync ... --regexmess "s,(.{9900}),$1\r\n,g"
+
+  On Unix
+  imapsync ... --regexmess 's,(.{9900}),$1\r\n,g'
+  
+Have also in mind that Exchange closes the connection after 10 errors
+encountered so you might also see "BYE Connection closed" errors from
+Exchange, which means Exchange leaves the session and says goodbye,
+come back later. Redo some sync then.
+
+On Linux, there is a good Python script in the tarball that can 
+fix several things that Office365 has issues with.
+
+Use it like this:
+
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
+
+It often does some miracles on messages.
+It's called fix_email_for_exchange because Office365 is Exchange,
+or at least started to be an Exchange server with the same issues.
+
+=======================================================================
+Q. From XXX to Office365 the flag Flagged does not seem to be well synced. 
+   What can I do?
+
+R. Use the following trick. Run imapsync twice, one with --regexflag
+   and one without, like this:
+
+1) imapsync ... --regexflag "s/\\Flagged//g"
+2) imapsync ...
+
+You can add --debugflags if you want to see what imapsync gets and 
+does in detail with flags.
+
+The magic of this trick is on ignoring the \Flagged flag on the first 
+sync and setting it on the second sync, with STORE instead of APPEND.
+This Exchange bug seems that Exchange gets and sets well the Flagged 
+flag with APPEND in IMAP but then it forgets it with other protocols;
+With STORE it sets and gets the "\Flagged" flag everywhere.
+
+Thanks to Dave Murray and Simon Savva for reporting and solving 
+this issue.
+
+======================================================================
+Q. How to migrate from or to Office 365 with an admin/authuser account?
+
+Note from Yago Torres Fernandez:
+(a working command using admin/authuser on host2 Office 365)
+
+  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com  ^
+               --password2 XXXX --ssl2  ^
+
+but previously in Office365, you must do something like the following, using Powershell:
+
+  Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all
+
+
+Note from Betsy Lawlor: You can use global modern authentication with two factor 
+on Exchange Online (M365) but you must have "AllowBasicAuthImap" on the admin account you are using to migrate the mail.
+
+Remark: PLAIN authentication is the only way to go with --authuser1 for now.
+So don't use --authmech1 SOMETHING with --authuser1 admin_user,
+it will not work.
+Same behavior with the --authuser2 option.
+
+Note from Rafael Alvarez Ballesteros:
+When you get an Office365 license you will receive an admin user to handle
+your licenses and products like admin@yourcompanyname.onmicrosoft.com.
+This account is the administrator account; some weeks or months ago Microsoft 
+has decided you need to use two-factor authentication by default, 
+so if two factors authentication is enabled you will no be able to sync 
+the mail (it will not connect to host2).
+OWA (I think this is the two-factor authentication) needs to be disabled 
+globally and enabled individually on the users you want to.
+One account can have the rights to access other mailboxes no matter if admin or 
+any other account but needs to have OWA disabled to be able to connect 
+to the office365 server.
+
+Note from Martin Paulucci:
+I had to remove the domain part for the user
+but not for the admin.  Example:
+
+  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated
+
+See also:
+http://www.linux-france.org/prj/imapsync_list/msg02203.html
+Subject: RE: [imapsync] Office 365 - 'Master User'?
+Date:    Mon, 1 Jun 2015 17:53:54 +0000
+
+
+=======================================================================
+Q. Couldn't create folder [trash] "Mailbox already exists".
+
+R. Some servers take care of character cases in folder names, 
+   some servers do not, like Exchange. Since not respecting the case can merge
+   two different folders into one then imapsync respects the case. 
+
+   For example, if a host1 server has a folder name called "trash"
+   and the host2 server already has a folder "Trash" or "TRASH"
+   then imapsync will try to create the folder "trash" on host2
+   because trash and Trash are different strings. But if host2
+   does not respect character case it will consider folder "trash"
+   already exists and will say it. That's the error message reported
+   by imapsync: "Mailbox already exists". This message comes from the server.
+
+   The folder creation fails but messages are well transferred in
+   so take a look at this warning, understand why it happens
+   and it should be fine most of the time.
+
+   To avoid this warning use --regextrans2 to map the folder names
+
+   imapsync ...  --regextrans2 "s/^trash$/Trash/"
+
+   If there are two folders Trash and trash on host1 then both
+   will be merged into only one Trash folder on host2. 
+   In case of the option --delete2 is used the regextans2 above becomes
+   mandatory, otherwise imapsync will sync messages from the
+   first Trash and then delete them when syncing trash.
+
+   If you want to avoid merging folders that are considered different 
+   on host1 but are considered the same on destination host2 because of
+   case sensitivities and insensitivities, use  --nomixfolders
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Old_Style_Web_Design.txt

@@ -1,20 +1,14 @@
 #!/bin/cat
-$Id: FAQ.Old_Style_Web_Design.txt,v 1.4 2019/11/25 13:08:31 gilles Exp gilles $
+$Id: FAQ.Old_Style_Web_Design.txt,v 1.9 2020/12/02 21:38:13 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.Old_Style_Web_Design.txt
 
 =======================================================================
-              Imapsync tips for Gmail accounts.
+              Imapsync and its ugly website style
 =======================================================================
 
-Questions answered in this FAQ are:
-
-Q. Why the imapsync website looks so old fashion?
-
-Now the questions again with their answers.
-
 =======================================================================
 Q. Why the imapsync website looks so old fashion?
 
@@ -29,28 +23,44 @@ Here are several reasons it stays like that for now:
 
 * I like to do things myself
 
-* I like static web
+* I am lazy
+
+* I like static web.
+  Although you could argue that I do use some dynamic things, like SSI. 
+  That's true, I use Server Side Include, aka SSI, denoted by the ".shtml" 
+  extention of some files, instead of the classical ".html" extention.
+  It's because I like dynamic web when the dynamic brings something that
+  can't be done easily with static files.
+  
 
 * I'm proud to be w3c compliant
   https://validator.w3.org/check?uri=https%3A%2F%2Fimapsync.lamiral.info
-  Very very few sites are w3c compliant, it's because all CMS frameworks aren't.
+  Very very few sites are w3c compliant, it's because nearly all CMS frameworks aren't.
+  Only true crazy people are w3c compliant seekers. I'm one of them.
+  If you know a CMS w3c compliant, tell me, it will be such a crazy news.
+  If you know any web site w3c compliant, tell me, it will be crazy news as well.
+  Update: https://contao.org/ is a CMS and it is w3c compliant! This is astonishing!
+  (Thanks to Markus Rupprecht for this crazy input!)
   
 * I prefer to spend more time on the product itself than the showcase
 
 * The product is a command line, a fancy website to sell a command line
   makes more disappointed buyers
 
-
 I agree I could do some sort of A/B experiment and see whether during
 a month I sell more imapsync on a brand new style B site than
 the current A one.
 
 
 Besides, don't misunderstand buyers. The surprising thing about complainers
-is that all are web designers, none is a user not in the webdesign branch. 
+is that all are web designers, none of them is a user not in the webdesign branch.
+
 I even received some phone call from a buyer saying: "I saw your website 
 and I told to myself: this can't be a fake one, the product must be a good one".
 
 Another folk like me:
 http://keyhut.com/pos.htm
-(well, not w3c compliant, since only mad people try to be w3c compliant)
+(well, not w3c compliant, he is not that mad)
+
+So maybe one day...
+

FAQ.d/FAQ.OnlineUI.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.OnlineUI.txt,v 1.20 2019/12/06 14:51:03 gilles Exp gilles $
+$Id: FAQ.OnlineUI.txt,v 1.27 2021/04/24 16:57:50 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -7,7 +7,6 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.OnlineUI.txt
 
 =====================================================================
    Imapsync tips about the online visual user interfaces
-   https://i005.lamiral.info/X/
    https://imapsync.lamiral.info/X/
 =====================================================================
 
@@ -15,9 +14,18 @@ Questions answered in this FAQ are:
 
 Q. How secure is the online visual user interface /X?
 
-Q. Will I have any issues with browser timing out? What happens
+Q. Does the online service store any sensitive information 
+   like my passwords?
+
+Q. Shall I have issues when the browser times out? What happens
    if the browser connection is closed for whatever reason?
 
+Q. Shall I have issues when the webserver times out? What happens
+   if the webserver closes the connection for whatever reason?
+
+Q. The sync stalls at the beginning, just after a line like:
+   "Host1: xxx says it has CAPABILITY for AUTHENTICATE LOGIN"
+   What is the problem?
 
 Now the questions again with their answers.
 
@@ -30,42 +38,41 @@ R0. Well, I don't know if asking the provider whether his online
 
 R1. Some figures
 
-Date of this report: 6 December 2019.
+Date of this report: 22 April 2021.
 
 The online imapsync service /X started 9 January 2017
-(1061 days of service).
+(1564 days of service).
 
-In average, /X has 50 users per day lunching in mean 6 
+On average, /X has 51 users per day lunching in mean 6
 different migrations, from just one launch to many (hundreds).
 
-The total volume /X transferred is around 101 TiB in more 
+The total volume /X transferred is around 210 TiB from nearly 
-than 219 thousands email imap migrations, 
+450 000 imap account migrations and 700 million email messages.
-340 millions email messages.
 
 R2. Pros & Cons
 
-The online imapsync service /X runs on https only, with a 
+The online imapsync service /X runs on HTTPS only, with a
 letsencrypt certificate, a certificate overall rated "A+" at
-https://www.ssllabs.com/ssltest/analyze.html?d=i005.lamiral.info
+https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
 
-Because of the https usage, what the users enter in their browser,
+Because of the HTTPS usage, what the users enter in their browser,
 the imap logins and passwords, can't be eavesdropped on the network.
 
-Imapsync itself takes care about encryption for the imap sessions, 
+Imapsync itself takes care of encryption for the imap sessions,
 if possible: It tries SSL first on port 993, then TLS on port 143
-if the servers announces TLS, then no encryption at all. 
+if the servers announce TLS, then no encryption at all.
 Concerning encryption, what is done with the source imap server host1
 is independent of what is done with the destination imap server host2.
 
-At the date of 6 December 2019, there is no security problem 
+On the date of 22 April 2021, there is no security problem
 detected or reported to me (Gilles LAMIRAL), so far.
 Feel free to attack the service and feel free to report any
 hole encountered. Have in mind I can watch what you try
-from the server side and take measure if the service suffers from
+from the server-side and take measures if the service suffers from
 your acts.
 
-As the owner of the service, it could have been 219 000 pairs of 
+As the owner of the service, it could have been 450 000 pairs of
-credentials collected and nearly 101 terabytes of email messages. 
+credentials collected and nearly 210 terabytes of email messages.
 I haven't kept them but I can't prove I haven't. It's just trust,
 like nearly every online service in the universe.
 
@@ -77,26 +84,43 @@ This default behavior is chosen like this because users of /X
 want their emails transferred, instead of being not transferred
 because of an incompetent imap server sysadmin.
 
-I admint that this part, checking imap ssl/tls certificates, 
+I admit that this part, checking imap ssl/tls certificates,
 could be improved from my side by including well known
 certificates directly in imapsync.
 
 If the imap servers don't honor ssl nor tls, then logins, passwords
 and everything will go clear text during the imap transfers.
-That's not good at all but what "comforts" me is that if the 
+That's not good at all but what "comforts" me is that if an
-imap servers do only clear text transfers, then it's also true 
+imap server does only clear text transfers, then it's also true
-for all imap sessions the accounts' owner encounters, 
+for all the imap sessions the account owner encounters,
-imapsync is just one of them.
+imapsync is just one of them; the imap servers are then always 
+unsecure for any imap software client.
+
 
 Last point, who could be sure that no cracker cracked the online
 hosts and that he isn't currently sniffing the credentials?
 
 No one, I'm not sure myself, even if I do take care of that
 possibility. So changing the imap accounts passwords after
-a sync is a safe and recommended practice!
+the sync is a safe and recommended practice!
 
 =====================================================================
-Q. Will I have any issues with browser timing out? What happens
+Q. Does the online service store any sensitive information 
+   like my passwords?
+
+No.
+The online passwords are kept by your browser. The goal is 
+that the next time you run a sync from your browser, all the 
+parameters are there without having to re-enter them.
+The save is done when you click on the "Sync or resync" green
+button. So, if you don't want your browser to keep your good password,
+enter a wrong one or empty the password field and click 
+on the "Sync or resync" green button, the wrong password 
+will be saved in your browser and the sync won't be done,
+since a good login is mandatory to do anything on your mailboxes.
+
+=====================================================================
+Q. Shall I have issues with the browser timing out? What happens
    if the browser connection is closed for whatever reason?
 
 R. It stops the imapsync process, ie, the sync is ended right away.
@@ -105,21 +129,21 @@ Further comments on this behavior.
 
 When using the /X interface there are three connections.
 One connection is the Browser/WebServer connection,
-the two others connections are the WebServer/ImapServers 
+the two other connections are the WebServer/ImapServers
 connections (imapsync stuff).
 
-If the Browser/WebServer connection is timeout or ended, 
+If the Browser/WebServer connection timeouts or ends,
 the imapsync sync is also ended immediately by the remote
-Apache https server. Technically, Apache sends a TERM signal
+Apache HTTPS server. Technically, Apache sends a TERM signal
 to the imapsync process, then wait some seconds before
 sending a KILL signal if it is still alive.
 
-You can relaunch a sync again with "Sync!" button, at any time.
+You can relaunch the sync with the "Sync!" button, at any time.
 If the "Sync!" button is gray/inactive then just reload
 the page (F5 or similar), and reenter the credentials.
 
-If the interface tells you that a sync is already going on,
+If the interface tells you that the sync is already going on,
-it may be that a sync is running from another browser or place.
+it may be that the sync is running from another browser or place.
 You can stop this sync with the "Abort!" button from any /X
 tab/window, even from another browser or place. To be able
 to abort with success, you have to give the same account
@@ -127,10 +151,35 @@ parameters, same credentials, or imapsync will ignore the demand.
 
 In other words, you can try safely to launch several parallel
 runs between the same mailboxes. Open a new tab/windows with /X,
-and start the exact same sync. It's safe, the /X will say, if any, that 
+and start the same sync. It's safe, the /X page will say that
-there is already a current sync running on them and it will present
+there is already one sync running and it will present
 the logfile running the sync like a "tail -f" command (isn't that magic?).
 
+=====================================================================
+Q. Shall I have issues when the webserver times out? What happens
+   if the webserver closes the connection for whatever reason?
+
+R. If the webserver closes the connection then usually it kills 
+   imapsync process and the imap connections as well.
+   To fix this issue, see the document INSTALL.OnlineUI.txt
+   and search for "Timeout" in it.
+   
+   Tip: If the connections close for huge folders and always after 
+   300 seconds or 60 seconds then it looks like a webserver timeout.
+   Apache 2.2 timeout is 300 seconds.
+   Apache 2.4 timeout is  60 seconds.
+
+=====================================================================
+Q. The sync stalls at the beginning, just after a line like:
+   "Host1: xxx says it has CAPABILITY for AUTHENTICATE LOGIN"
+   What is the problem?
+
+R. I've seen this issue on /X with the imap server
+   CommuniGate Pro IMAP Server 6.0.11
+   The issue looks related to special characters in the password.
+   Solution for now: change the password, keep only
+   standard (ASCII) alphanumeric characters ABC-YZ abc-yz 012-89.
+
 
 =====================================================================
 =====================================================================

FAQ.d/FAQ.Passwords_on_Mac.txt

@@ -0,0 +1,38 @@
+#!/bin/cat
+$Id: FAQ.Passwords_on_Mac.txt,v 1.1 2020/02/12 11:25:48 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+
+
+=======================================================================
+              Imapsync issues with passwords on Mac OS X.
+=======================================================================
+
+
+=======================================================================
+Q. On Mac, some passwords contain some weird *()",;&~ characters. 
+   Login fails.
+
+R1. Enclose the password within single-quotes in the imapsync
+    command line:
+
+    imapsync ... --password1 'passw*()",;&rd~'
+
+CAVEAT: Be careful to use quotes and double-quotes characters encoded
+from the ASCII table. How to be sure of that? 
+Mac OS X has a "use smart quotes" option. This is particularly handy 
+when typesetting and the system automatically substitutes opening and 
+closing quotes. It isn't handy when editing the imapsync command line.
+
+Disable "use smart quotes and dashes" in the Keyboard configuration,
+located in the "Text" tab. It will fix this unfortunate character 
+transformation. You can put it back later.
+
+R2. Change the password in order to keep only f...ing normal characters,
+    ie, alphanumerics like aAzZ-_09, and minus and underscore.
+    Make it long and random if strong security is your concern.
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Passwords_on_Unix.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Passwords_on_Unix.txt,v 1.6 2019/04/02 17:19:21 gilles Exp gilles $
+$Id: FAQ.Passwords_on_Unix.txt,v 1.7 2020/12/04 11:46:54 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -12,19 +12,23 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
 
 
 =======================================================================
-Q. On Unix, some passwords contain some weird *(),;&~ characters. 
+Q. On Unix, some passwords contain some weird *()$,";&~ characters.
    Login fails.
 
 R1. Enclose the password within single-quotes in the imapsync
     command line:
 
-    imapsync ... --password1 'passw*(),;&rd~'
+    imapsync ... --password1 'passw*()`"$,;&rd~'
 
-R2. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
+
+R2. Change the password to keep only f...ing normal characters.
+    Make it long and random if strong security is your concern.
+
+R3. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
     try also using double-quotes within single-quotes. It will enclose
     the password within double-quotes in the imap LOGIN command:
 
-    imapsync ... --password1 '"passw*(),;&rd~"'
+    imapsync ... --password1 '"passw*()`$,;&rd~"'
 
 Old Mail::IMAPClient 3.37 is a bad horse having this issue solved
 by this double-quotes trick.
@@ -32,9 +36,5 @@ fixed in 3.38 https://metacpan.org/changes/distribution/Mail-IMAPClient
 version 3.38: Tue Feb  9 02:48:21 UTC 2016
         - rt.cpan.org#110273: failure to quote password values
           (regression introduced in 3.36 via fix for rt.cpan.org#100601)
-
-R3. Change the password to keep only f...ing normal characters.
-    Make it long and random if strong security is your concern
-
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Principles.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Principles.txt,v 1.5 2019/09/05 13:33:46 gilles Exp gilles $
+$Id: FAQ.Principles.txt,v 1.10 2021/04/18 13:00:00 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -13,28 +13,36 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Principles.txt
 
 Questions answered in this FAQ are:
 
-Q. What is the bandwidth used by imapsync?
+Q. Give me some design ideas of imapsync
-
 
 Now the questions again with their answers.
 
 =======================================================================
-Q. What is the bandwidth used by imapsync?
+Q. Give me some design ideas of imapsync
 
-R. From the host where imapsync runs, it opens two imap connections, 
+R.
-   one with the source account at host1, one with the destination 
-   account at host2.
 
-   So, the global bandwidth used is twice the volume of the source 
+* No configuration file
-   account, one volume to download the messages from host1, 
+* Stateless
-   one volume to upload those messages to host2.
+* Rsync in mind
+* Reliable
+* Robuste
+* The user is not an expert, help him with:
+  * Good behavior by default. 
+  * No harm by default. 
+  * Prefer fixing an issue in the code than documenting the issue
+  * Inline help suggestions
 
-   If the host2 is already filled with the messages, imapsync doesn't
+Consequences of those design ideas:
-   transfer them and then the volume transferred is small, just the IMAP
+
-   commands to identify the messages on both sides.
+* Stateless: 
+
+   * A sync can be done by another tool before or after imapsync, it is
+   safe as long as the other tools don't modify the messages' headers 
+   "Message-Id:" and "Received:", or at least "Message-Id:".
+
+   * Ready to run in a container.
 
-   There is no local cache of the email messages, except when a 
-   message is very big, it is temporarly saved locally. 
 
 
 =======================================================================

FAQ.d/FAQ.Release_Checklist.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Release_Checklist.txt,v 1.16 2020/01/04 09:35:36 gilles Exp gilles $
+$Id: FAQ.Release_Checklist.txt,v 1.18 2020/01/06 10:47:02 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -32,13 +32,14 @@ Checklist before release a new release:
 - /X verify under noscript with firefox
 - /X verify under private with chrome
 
-- Check the binaries imapsync.exe imapsync_64bit.exe with
+- Check the binaries imapsync.exe imapsync_32bit.exe with
   https://www.virustotal.com/
   https://www.metadefender.com/
 
 
 - make dist
 - verify on Windows the zip extraction and simple execution of imapsync_example.bat
+- verify binary version matches imapsync script version
 - make publish
 - verify VERSION is uploaded
 

FAQ.d/FAQ.SSL_errors.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.SSL_errors.txt,v 1.9 2019/12/11 15:38:37 gilles Exp gilles $
+$Id: FAQ.SSL_errors.txt,v 1.14 2021/04/19 11:47:44 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -21,7 +21,11 @@ Q. What can I do to avoid those "SSL read/write errors"?
 
 Q. SSL connect attempt failed SSL
    routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
+   or 
+   SSL connect attempt failed SSL
+   SSL routines:ssl_choose_client_version:unsupported protocol
 
+Q. How to see the certificate and identify problems in it?
 
 Now the questions again with their answers.
 
@@ -32,29 +36,29 @@ Q. What are the errors
   DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
 
 
-R1.Like they claim, those errors are SSL errors. SSL is not directly
+R1. As they claim, those errors are SSL errors. SSL is not directly
-   done by imapsync but by an underlying Perl module called
+done by imapsync but by an underlying Perl module called
-   IO::Socket::SSL. Those errors arise sometimes and sometimes
+IO::Socket::SSL. Those errors arise sometimes and sometimes
-   they form a serie that ends with imapsync auto-abortion.
+they form a series that ends with imapsync auto-abortion.
-   Those errors happen with some hosts but not with others,
+Those errors happen with some hosts but not with others,
-   it's often Exchange or Office365. I don't know what exactly happens.
+it's often Exchange or Office365. I don't know what exactly happens.
-   Those errors happen more often on Windows than on Linux.
+Those errors happen more often on Windows than on Linux.
 
 
 =======================================================================
 Q. What can I do to avoid those "SSL read/write errors"?
 
 R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
-    Those errors don't appear with recent releases, post 1.836
+Those errors don't appear with recent releases, post 1.836
 
 R1. Remove all ssl/tls encryption
 
   imapsync ...   --nossl1 --notls1 --nossl2 --notls2
 
 R2. If you don't want to quit encryption, rerun imapsync until the 
-    complete sync is over. Those errors are not at the same place 
+complete sync is over. Those errors are not at the same place 
-    each time, so imapsync will sync remaining messages at each run
+each time, so imapsync will sync the remaining messages at each run
-    until none remains.
+until none remains.
 
 R3. Run imapsync on a Linux machine, a VM is ok, there are less
     SSL errors on Unix.
@@ -69,12 +73,16 @@ R5. Set up a ssltunnel proxy to the host.
 =======================================================================
 Q. SSL connect attempt failed SSL
    routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
+   or 
+   SSL connect attempt failed SSL
+   SSL routines:ssl_choose_client_version:unsupported protocol
+
 
 R1. Use: 
 
-   imapsync ... --sslargs1 'SSL_cipher_list=DEFAULT' 
+   imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
    or    
-   imapsync ... --sslargs2 'SSL_cipher_list=DEFAULT' 
+   imapsync ...  --sslargs2 SSL_cipher_list=DEFAULT
 
    depending on where the error occurs, host1 or host2 or both.
 
@@ -82,13 +90,27 @@ R2. If it doesn't work, I let you try other things,
     I quote the "SSL_version" section of
     https://metacpan.org/pod/IO::Socket::SSL (Module version: 2.066)
 
+    imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
     imapsync ...  --sslargs1 SSL_version=SSLv2
+    imapsync ...  --sslargs1 SSL_version=SSLv23
+    imapsync ...  --sslargs1 SSL_version=SSLv3
+    imapsync ...  --sslargs1 SSL_version=TLSv1
+    imapsync ...  --sslargs1 SSL_version=TLSv1_1
     imapsync ...  --sslargs1 SSL_version=TLSv1_2
+    imapsync ...  --sslargs1 SSL_version=TLSv1_3
 
-    SSLv2 and TLSv12 are just examples depending on your context
+Those examples are for host1. For host2, use --sslargs2 instead.
-    (--ssl1 or --tls1, and also the imap server encryption scheme)
+Feedback on what worked for you is welcome!
 
-Feedback on what worked for you (and possibly hy) is welcome!
+A loop to check every version and print the good ones:
+
+for v in SSLv2 SSLv23 SSLv3 TLSv1 TLSv1_1 TLSv1_2 TLSv1_3; do
+        imapsync ...  --sslargs1 SSL_version=$v && GOOD="$GOOD $v"
+done
+echo "$GOOD"
+
+I reproduce below the documentation of the underlying Perl 
+module IO::Socket::SSL used by imapsync:
 
 https://metacpan.org/pod/IO::Socket::SSL
 ...
@@ -124,10 +146,78 @@ In this case setting the version
 to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.
 
 
+=======================================================================
+Q. How to see the certificate and identify problems in it?
+
+R. Use the command openssl like this:
+
+  echo | openssl s_client -crlf  -connect imap.gmail.com:993
+
+  echo | openssl s_client -crlf  -connect test1.lamiral.info:993
+
+and examine carefully the content, the  "verify return:" lines,
+the chain. Sometimes, the server certificate is ok but not the whole 
+chain of certificates so the certification fails.
+
+Here is an example.
+
+One of the certificate is expired:
+
+echo | openssl s_client -crlf -connect test1.lamiral.info:993
+CONNECTED(00000003)
+depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
+verify return:1
+depth=1 C = US, O = Let's Encrypt, CN = R3
+verify return:1
+depth=0 CN = test1.lamiral.info
+verify error:num=10:certificate has expired
+notAfter=Apr 11 10:14:05 2021 GMT
+verify return:1
+depth=0 CN = test1.lamiral.info
+notAfter=Apr 11 10:14:05 2021 GMT
+verify return:1
+---
+Certificate chain
+ 0 s:/CN=test1.lamiral.info
+   i:/C=US/O=Let's Encrypt/CN=R3
+ 1 s:/C=US/O=Let's Encrypt/CN=R3
+   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
+---
+Server certificate
+-----BEGIN CERTIFICATE-----
+MIIFKjCCBBKgAwIBAgISBHYZCE3qSTIlvq97HI5TpBeAMA0GCSqGSIb3DQEBCwUA
+MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
+EwJSMzAeFw0yMTAxMTExMDE0MDVaFw0yMTA0MTExMDE0MDVaMB0xGzAZBgNVBAMT
+EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
+ggEBAMUTJVdrTl86nDI2yO6Vz5l1qxMMPqJylQcgi9vDHpwsnUq5HGPv+qZNhM69
+...
+
+After an complete server update ("apt update && apt upgrade"):
+
+echo | openssl s_client -crlf -connect test1.lamiral.info:993
+CONNECTED(00000003)
+depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
+verify return:1
+depth=1 C = US, O = Let's Encrypt, CN = R3
+verify return:1
+depth=0 CN = test1.lamiral.info
+verify return:1
+---
+Certificate chain
+ 0 s:/CN=test1.lamiral.info
+   i:/C=US/O=Let's Encrypt/CN=R3
+ 1 s:/C=US/O=Let's Encrypt/CN=R3
+   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
+---
+Server certificate
+-----BEGIN CERTIFICATE-----
+MIIFKTCCBBGgAwIBAgISBD4QN3cfB1JpTm75oVrkkAElMA0GCSqGSIb3DQEBCwUA
+MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
+EwJSMzAeFw0yMTAzMTIxODQxMTJaFw0yMTA2MTAxODQxMTJaMB0xGzAZBgNVBAMT
+EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
+ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
+...
+
+
 =======================================================================
 =======================================================================
-
-
-
-
-

FAQ.d/FAQ.Security.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-# $Id: FAQ.Security.txt,v 1.19 2018/09/13 17:53:45 gilles Exp gilles $
+# $Id: FAQ.Security.txt,v 1.22 2020/06/30 17:32:39 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -32,7 +32,7 @@ R1. The online UI does TLS/SSL imap connections if the imap
     servers support TLS/SSL. If you are concerned by security then 
     using the .bat file or .sh on your computer should be more secure 
     since you can examine and secure it by yourself, no matter high is
-    your paranoïd spirit. The online UI security is mine, I am
+    your paranoid spirit. The online UI security is mine, I am
     concerned by security, not to the upmost high level possible 
     but I won't give you direct access to the host to discover my level.
     With a good guy spirit, feel free to try to break the online UI 
@@ -159,8 +159,8 @@ b OK 73 good day (Success)
 The client part you have to type is 
 a LOGIN ... 
 b LOGOUT
-while replacing ... by your credentials values.
+while replacing ... by your credentials values,
-
+other lines are the server responses.
 
 =======================================================================
 Q.How can I test an tls imap connection without imapsync?
@@ -200,6 +200,19 @@ to help the ssl software verifying the server certificate.
 The file ca-certificates.crt may be elsewhere on your system, even 
 named differently.
 
+The imap server certificates are not checked for authenticity 
+(by default) because too many imap servers are crappy configured 
+regarding certified certificates.
+
+This default behavior is chosen like this because users
+want their emails transferred, instead of being not transferred 
+because of an incompetent imap server sysadmin.
+
+I admint that this part, checking imap ssl/tls certificates, 
+could be improved from my side by including well known 
+certificates directly in imapsync.
+Drop me a note to encourage me.
+
 =======================================================================
 Q: How to have an imaps server?
 

FAQ.d/FAQ.SmarterMail.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.SmarterMail.txt,v 1.14 2019/10/15 14:57:19 gilles Exp gilles $
+$Id: FAQ.SmarterMail.txt,v 1.15 2021/04/30 13:02:17 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,6 +10,39 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.SmarterMail.txt
     Imapsync tips for SmarterMail. Specific issues and solutions.
 =======================================================================
 
+https://imapsync.lamiral.info/README
+...
+     --prefix1      str  : Remove prefix str to all destination folders,
+                           usually "INBOX." or "INBOX/" or an empty string "".
+                           imapsync guesses the prefix if host1 imap server
+                           does not have NAMESPACE capability. So this option
+                           should not be used most of the time.
+     --prefix2      str  : Add prefix to all host2 folders. See --prefix1
+
+     --sep1         str  : Host1 separator. This option should not be used
+                           most of the time.
+                           Imapsync gets the separator from the server itself,
+                           by using NAMESPACE, or it tries to guess it
+                           from the folders listing (it counts
+                           characters / . \\ \ in folder names and choose the
+                           more frequent, or finally / if nothing is found.
+     --sep2         str  : Host2 separator. See --sep1
+
+
+
+ --noabletosearch    : Makes --minage and --maxage options use the internal 
+                       dates given by a FETCH imap command instead of the 
+                       "Date:" header. Internal date is the arrival date 
+                       in the mailbox.
+                       --noabletosearch equals --noabletosearch1 --noabletosearch2
+
+ --noabletosearch1   : Like --noabletosearch but for host1.
+ --noabletosearch2   : Like --noabletosearch but for host2.
+
+ --useheader    str  : Use this header to compare messages on both sides.
+                       Example: Message-ID or Subject or Date.
+ --useheader    str    and this one, etc.
+
 
 =======================================================================
 Q. Synchronizing from SmarterMail to XXX
@@ -17,12 +50,12 @@ Q. Synchronizing from SmarterMail to XXX
 On Unix:
 imapsync --host1 imap.d1.org --user1 joe --password1 secret1 \
          --host2 imap.d2.org --user2 joe --password2 secret2 \
-         --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch
+         --prefix1 "" --sep1 "/" --useheader Message-Id --noabletosearch
 
 On Windows:
 imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^
          --host2 imap.d2.org --user2 joe --password2 secret2  ^
-         --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch
+         --prefix1 "" --sep1 "/" --useheader Message-Id --noabletosearch
 
 
 =======================================================================
@@ -31,12 +64,12 @@ Q. Synchronizing from XXX to SmarterMail
 On Unix:
 imapsync --host1 imap.d1.org --user1 joe --password1 secret1  \
          --host2 imap.d2.org --user2 joe --password2 secret2  \
-         --sep2 "/" --prefix2 "" --useheader Message-Id --noabletosearch
+         --prefix2 "" --sep2 "/" --useheader Message-Id --noabletosearch
 
 On Windows:
 imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^
              --host2 imap.d2.org --user2 joe --password2 secret2 ^
-             --sep2 "/" --prefix2 "" --useheader Message-Id --noabletosearch
+             --prefix2 "" --sep2 "/" --useheader Message-Id --noabletosearch
 
 
 =======================================================================

FAQ.d/FAQ.Two_Ways_Sync.txt

@@ -0,0 +1,65 @@
+$Id: FAQ.Two_Ways_Sync.txt,v 1.4 2021/02/01 15:43:33 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Two_Ways_Sync.txt
+
+
+=======================================================================
+             Two ways sync with Imapsync 
+=======================================================================
+
+
+Questions answered in this FAQ are:
+
+Q. Can Imapsync do a good "two ways" sync? 
+   No. Why?
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Can Imapsync do a good "two ways" sync? 
+   No. Why?
+
+R. Imapsync can't do good two ways syncs.
+
+A good "two ways" sync is impossible with imapsync because imapsync is
+stateless.
+
+Each time imapsync runs, it considers messages and folders as if it
+were the first time it encounters them. Imapsync looks at messages,
+flags, and folders as they are now, not considering what they were
+before. Imapsync has no memory outside the current running sync.
+
+So now, why a stateless behavior cannot handle well a two ways sync
+between an account A and an account B?
+
+The problem arises with deletions, messages deletions, folders
+deletions, or movings, messages movings across folders, folders
+movings, and also folders renamings. Deletions and moves are ambiguous
+changes when combined with creations on the opposite side.
+ 
+For example, when a message is deleted from A by a user, imapsync
+cannot know whether it is a message deleted from A that has to be
+deleted in B (what the user did) or a missing message from B that has
+to be copied to A.
+
+But if you know the answer yourself, that missing messages on one side
+A are deleted messages that have to be deleted on the other side then
+run a sync with the --delete2 option from A to B.
+
+If you know that the missing messages on A are missing messages from B
+that has to be copied to A then run a sync from B to A.
+
+If you know it's a mixed scenario then you are in trouble and so you
+end up with a not very good "two ways" sync. I suggest avoiding
+deletions in that case, which is the default imapsync behavior.
+
+With a two ways sync, the mailbox user is very surprised and
+disapointed when his actions (deletions, renamings, or movings) come
+back.
+
+=======================================================================
+=======================================================================
+

FAQ.d/FAQ.Use_cache.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Use_cache.txt,v 1.9 2019/09/13 10:43:06 gilles Exp gilles $
+$Id: FAQ.Use_cache.txt,v 1.11 2021/04/22 13:20:18 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -7,7 +7,7 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
 
 
 =======================================================================
-               Imapsync  --usecache option
+               Imapsync  --usecache or --useuid options
 =======================================================================
 
 Questions answered in this FAQ are:
@@ -16,6 +16,9 @@ Q. What is --usecache good for?
 
 Q. What is the technical principle of --usecache?
 
+Q. On Windows, with --useuid and after a reboot or a new login, a new
+   run generates duplicates. What can I do to avoid and fix that?
+
 Q. On Windows, with --useuid or --usecache a problem occurs with long
    nested folder names. The error message is:
    "No such file or directory; The filename or extension is too long"
@@ -23,14 +26,14 @@ Q. On Windows, with --useuid or --usecache a problem occurs with long
 Q. Inode problem with --usecache or --useuid on Linux
 
 
-Now the questions again and their answers
+Now the questions again and their answers.
 
 =======================================================================
 Q. What is --usecache good for?
 
 R. The option --usecache is useful to speed up next syncs with --usecache.
    The first sync with --usecache between two imap accounts is as low 
-   as the first sync without --usecache.
+   as the first sync without --usecache, the next syncs are faster.
 
 =======================================================================
 Q. What is the technical principle of --usecache?
@@ -49,6 +52,38 @@ R. Option --usecache creates a empty file per email message
    where <tmpdir> is the temporary directory given by the --tmpdir option
    or the one by default, /tmp on Unix, variable on Windows.
    
+=======================================================================
+Q. On Windows, with --useuid and after a reboot or a new login, a new
+   run generates duplicates. What can I do to avoid and fix that?
+
+R. The cache is build in a temporary directory. Imapsync uses a system
+call to get this temporary directory, it is not hardcoded and it
+is system dependent, the call garanties a write access. Sometimes on 
+Windows this directory name changes upon reboot or for any other reason. 
+Anyway, the temporary directory used by imapsync can be set by the 
+option --tmpdir and that's the way to avoid this issue. 
+The new problem is that the temporary directory must be writable, which
+is not a big deal to found.
+
+Use for example either a) b) or c)
+
+a) The current directory:
+
+  imapsync ... --tmpdir .
+
+b) The root directory:
+
+  imapsync ... --tmpdir /
+
+c) Any directory you want, like:
+
+  imapsync ... --tmpdir "C:\TEMP"
+
+Now, to fix the duplicates on host2, have a run with the option
+--delete2duplicates
+
+  imapsync ... --delete2duplicates
+
 
 =======================================================================
 Q. On Windows, with --useuid or --usecache a problem occurs with long

FAQ.d/FAQ.Various_Software_Servers.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Various_Software_Servers.txt,v 1.13 2019/07/04 09:26:28 gilles Exp gilles $
+$Id: FAQ.Various_Software_Servers.txt,v 1.19 2021/01/26 14:05:23 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -18,6 +18,13 @@ Q. From or to Zimbra
 R. Read the FAQ.Zimbra.txt 
    https://imapsync.lamiral.info/FAQ.d/FAQ.Zimbra.txt
 
+=======================================================================
+Q. From Emailchemy (https://weirdkid.com/emailchemy/)
+
+R. Use option  --noabletosearch :
+
+  imapsync ... --noabletosearch
+
 =======================================================================
 Q. From or to HMailServer version 4.4.1.
 
@@ -51,60 +58,10 @@ Parallells Plesk Panel for Windows requires --sep2 / --prefix2 ""
 Parallells Plesk Panel for Linux works with default parameters.
 
 =======================================================================
-Q. I'm migrating from WU to Cyrus, and the mail folders are under
+Q. Migrating from or to the Cyrus imap server
-   /home/user/mail but the tool copies everything in /home/user, how
-   can i avoid that?
 
-Two solutions:
+R. Read the dedicated 
-
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
-R. Use
-   imapsync ... --include '^mail'
-
-R. or (better)
-   imapsync ... --subscribed --subscribe
-
-
-=======================================================================
-Q. I'm migrating from WU to Cyrus, and the mail folders are under
-   /home/user/mail directory.  When imapsync creates the folders in
-   the new cyrus imap server, it makes a folder "mail" and below that
-   folder puts all the mail folders the user have in /home/user/mail,
-   i would like to have all those folders directly under INBOX.
-
-R. Use 
-   imapsync ... --regextrans2 's/^mail/INBOX/' --dry
-   look at the simulation and if all transformations seem
-   good then remove the --dry option.
-
-
-=======================================================================
-Q. Migrating from Groupwise to Cyrus
-
-R. By Jamie Neil:
-
-I eventually managed to get the mail to migrate without errors using the 
-following options:
-
---maxage 3650
-  - some messages just don't seem to want to transfer and produce the 
-    perl errors I mentioned before. This prevents the errors, but the
-    bad messages don't transfer.
-
-Even though the mail migrated OK, there are a couple of gotchas with 
-Groupwise IMAP:
-
-1) Some of the GW folders are not real folders and are not available
-to IMAP, the main problem one being "Sent Items". I could find no way
-of coping the contents of these folders. The nearest I got was to
-create a "real" folder and copy/move the sent items into it, but
-imapsync still didn't see the messages (I think because there is
-something funny about the reported dates/sizes).
-
-It think this problem has been rectified in GW6.5.
-
-2) Version 6.0.1 of the Groupwise Internet Connector sucks. I was 
-getting server aborts when I pushed it a bit hard! I eventually had to
-upgrade to 6.0.4 which seems to be a lot more stable.
 
 =======================================================================
 Q. Migrating from iPlanet Messaging Server
@@ -131,15 +88,6 @@ You have to extract users logins and passwords in a csv file.
 See the "HUGE MIGRATION" section in the README file.
 
 
-======================================================================
-Q: From MailEnable 1.75
-R: --sep1 "/" --prefix1 ""
-
-Q: From MailEnable 2.2
-R: --sep1 "." --prefix1 ""
-
-Q: To MailEnable
-R: --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
 
 ======================================================================
 Q. From GMX IMAP4 StreamProxy
@@ -174,53 +122,6 @@ Q. From or to QQMail IMAP4Server
 
 R. imapsync ... --noabletosearch1
 
-======================================================================
-Q. From FirstClass to XXX
-   http://www.firstclass.com/
-
-R. Migrating from FirstClass is not easy because FirstClass, strangely, 
-   does not show all messages via IMAP. To make it show all messages,
-   a trick, painful to follow by hand, is moving emails 
-   out and back in, for each folder. May be it can be done by a script.
-
-  FirstClass releases prior to release 12 do not shows the "Sent" 
-  folder in IMAP but FirstClass release 12 shows it.
-  I advice you to upgrade to FirstClass release 12 before leaving it
-  with imapsync or another imap tool.
-
-Here is a command line used to migrate from FirtClass release 12:
-
-  imapsync ... \
-    --tmpdir /var/tmp --usecache \
-    --useheader Message-ID \
-    --idatefromheader \
-    --addheader \
-    --regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
-    --regextrans2 "s/[\^]/_/g"       \
-    --regextrans2 "s/['\"\\\\]/_/g"  \
-    --regextrans2 "s,&AC8-,-,g" \
-    --regextrans2 "s,&APg-,oe,g" 
-
-On Windows:
-  imapsync.exe ... ^
-    --automap ^
-    --usecache ^
-    --useheader Message-ID ^
-    --idatefromheader ^
-    --addheader ^
-    --regextrans2 "s,(/|^) +,$1,g" ^
-    --regextrans2 "s, +(/|$),$1,g" ^
-    --regextrans2 "s/[\^]/_/g" ^
-    --regextrans2 "s/['\\]/_/g" ^
-    --regextrans2 "s,^&AC8-,-,g"  ^
-    --regextrans2 "s,^&APg-,oe,g"
-
-
-Special thanks to Kristian Wind and Joey Alexander for helping me
-writing this FAQ item. 
-See also this worth reading discussion in a Zimbra forum:
-http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
-
 ======================================================================
 Q. From XXX to FTGate
 

FAQ.d/FAQ.Virus.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Virus.txt,v 1.6 2019/12/06 09:43:52 gilles Exp gilles $
+$Id: FAQ.Virus.txt,v 1.9 2020/01/21 21:35:53 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -17,14 +17,15 @@ Q. My virus scanner claims imapsync.exe is a virus/malware/trojan etc.
 R1. Yes, I find the same sometimes with few antivirus, some report that 
    imapsync might have a trojan in it.
    Most virus scanners say imapsync.exe and  is ok.
-   I've done this test on imapsync.exe release 1.727 on the two following
+   I've done this test on imapsync.exe releases 
+   1.977, 1.945, 1.836 on the two following
    meta-virus-scanners:
    https://www.virustotal.com/
    https://www.metadefender.com/
 
 R2. Explanation: 
     It may come from the fact that imapsync checks whether there is a
-    new realease available at http://imapsync.lamiral.info/VERSION
+    new release available at http://imapsync.lamiral.info/VERSION
     It's explained here:
     https://imapsync.lamiral.info/#NUMBERS
 
@@ -40,6 +41,20 @@ R4. Using online meta-virus-scanners
     https://www.virustotal.com/
     https://www.metadefender.com/
 
+
+    Release 1.977
+    
+    imapsync.exe (64bit) release 1.977 is reported as not clean:
+    * https://www.virustotal.com/gui/file/8fd16a5eb2aa4d9a037a528654cc0ecdc63936955d289811dee31c6af474f4b0/detection
+      SecureAge APEX Malicious Jiangmin Backdoor.Kasidet.qt
+    
+    imapsync_32bit.exe release 1.977 is reported as totally clean:
+    https://www.virustotal.com/gui/url/94c89ef31190050bebc522f64e32d3f825ef0418cfc424c5847495b9857d57bf/detection
+
+    imapsync_bin_Darwin release 1.977 is reported as totally clean:
+    * https://www.virustotal.com/gui/file/41f107f4f7c86b15c392aef3c69a7f285f3b2a5f02328a1342903eddc5a5a06f/details
+    * https://metadefender.opswat.com/results#!/file/bzIwMDEwNUJKb2dXa1N3eWVJSDFoZ1d5clBrZ1U/regular/overview
+
     Release 1.945
     
     imapsync.exe release 1.945 is reported as totally clean:

FAQ.d/FAQ.XOAUTH2.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.XOAUTH2.txt,v 1.16 2019/07/31 22:58:36 gilles Exp gilles $
+$Id: FAQ.XOAUTH2.txt,v 1.17 2021/01/21 13:25:02 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,9 +10,47 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
   Imapsync tips to use XOAUTH2 authentication (Gmail) and old XOAUTH 
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal gmail account? (Quick answer: not yet)
+
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal Office365 account? (Quick answer: not yet)
+
+Q. Is XOAUTH2 authentication available with imapsync to globally 
+   authenticate gmail users, ie as an admin?
+
+Q. Imapsync XOAUTH2 fails with the following message, how to fix that?
+
+Q. How to use XOAUTH2 via a json file to globally authenticate gmail users? 
+
+Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users?
+
+Q. How to use a proxy with XOAUTH2 authentication?
+
+Q. How to use old XOAUTH to globally authenticate gmail users?
+
+
+Now the questions again with their answers.
 
 =======================================================================
-Q. Is XOAUTH2 authentication available with imapsync?
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal gmail account? (Quick answer: not yet)
+
+R. No. But I'm working on that. 
+   XOAUTH2 is a mess or at least, each time I look at it, it leaves 
+   my brain like a mess.
+
+=======================================================================
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal Office365 account? (Quick answer: not yet)
+
+R. No. But I'm working on that too.
+
+=======================================================================
+Q. Is XOAUTH2 authentication available with imapsync to globally 
+   authenticate gmail users, ie as an admin?
 
 R. Yes, but XOAUTH2 has been really tested on Unix systems, 
    less profund on Windows but it should work.
@@ -172,7 +210,7 @@ environment variable, if it isn't already set. Example:
 
 
 =======================================================================
-Q. How to use XOAUTH to globally authenticate gmail users?
+Q. How to use old XOAUTH to globally authenticate gmail users?
 
 R0. XOAUTH is considered obsolete and superseded by XOAUTH2
     Anyway the manage part might be the same (I don't know).

FAQ.d/FAQ.Yahoo.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Yahoo.txt,v 1.8 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Yahoo.txt,v 1.10 2021/04/20 21:47:02 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -14,35 +14,18 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Yahoo.txt
 =======================================================================
 Q. Synchronizing from Yahoo to XXX
 
-R. Use --host1 imap.mail.yahoo.com --ssl1
+R. Use --host1 imap.mail.yahoo.com 
 
-./imapsync \
+Example:
+
+  ./imapsync \
     --host1 imap.mail.yahoo.com \
     --user1 billy \
     --password1 secret \
-    --ssl1 \
     --host2 XXX \
     --user2 billy \
     --password2 secret \
-    --noabletosearch --errorsmax 10000 
+    --noabletosearch 
-
-SSL is mandatory for yahoo since November 2011.
-
-Why increasing --errorsmax to 10000?
-Because during the sync you may encounter several lines like this one:
-- msg Inbox/12600 {0} S[36012] F[] I[27-Nov-2015 07:08:35 +0000] could not be fetched: 190 NO [UNAVAILABLE] UID FETCH Server error while fetching messages
-- msg Inbox/12600 skipped.
-I don't know why it happens, why Yahoo server says 
-"NO [UNAVAILABLE] UID FETCH Server error while fetching messages"
-Usually another run syncs the message successfully.
-
-Why --noabletosearch option?
-Yahoo IMAP does not honor well the SEARCH imap command (Dec 2017).
-So don't use the --search option.
-You can use --minage or/and --maxage but then add also --noabletosearch
-
-Yahoo bandwidth per connection is relatively low, 50 KiB/s to 500 KiB/s max, usually 100 KiB/s.
-Consider using --useuid to speed up large accounts.
 
 
 You also need to go to Yahoo, security and enable 

FAQ.d/FAQ.Zimbra.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Zimbra.txt,v 1.6 2019/06/25 00:15:16 gilles Exp gilles $
+$Id: FAQ.Zimbra.txt,v 1.11 2021/04/18 09:15:45 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -15,14 +15,56 @@ https://wiki.zimbra.com/wiki/Guide_to_imapsync
 It's obsolete and it will give you, and me, more difficulties than 
 doing a standard sync without additional options.
 
+Questions answered in this FAQ are:
+
+Q. How to migrate from Zimbra to XXX?
+
+Q. How to migrate from Zimbra with an admin account?
+
+Q. I got this "NO APPEND failed" error:
+could not append ( Subject:... Flags:[\Flagged \Seen EmailswithStickyNotes:FLAG51135] ) 
+to folder XXX: 316 NO APPEND failed
+What can I do?
+
+Q. Zimbra disconnects without reason
+   Could not select: * BYE ... Zimbra IMAP4rev1 server closing connection
+
+
+Now the questions again with their answers.
+
 
 =======================================================================
-Q. From Zimbra to XXX
+Q. How to migrate from Zimbra to XXX?
 
-  imapsync ... \
+The basic default (only credendials) should work fine with Zimbra.
-        --exclude "Conversation Action Settings" \
+
-        --exclude "Quick Step Settings" \
+
-        --exclude "News Feed"
+=======================================================================
+Q. How to migrate from Zimbra with an admin account?
+
+R. Use:
+
+  imapsync ... --user1 "normal_user" --authuser1 "admin_user"  --password1 "admin_user_password"
+
+To setup or use a Zimbra admin user see:
+https://zimbra.github.io/adminguide/8.8.9/index.html#_administrator_accounts
+
+Thanks to Richard Street from thinkround for this tip.
+
+=======================================================================
+Q. I got this "NO APPEND failed" error:
+could not append ( Subject:... Flags:[\Flagged \Seen EmailswithStickyNotes:FLAG51135] ) 
+to folder XXX: 316 NO APPEND failed
+What can I do?
+
+R1.
+https://zimbra.github.io/adminguide/8.8.10/index.html
+Flagging
+Users can create flags and assign them to messages, contacts, and files in Briefcase folders.
+In the Admin Console, see COS -> Features -> Mail Features container
+
+Is Flagging on on host2? 
+If yes, see https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
 
 =======================================================================
 Q. Zimbra disconnects without reason
@@ -37,17 +79,5 @@ R. The cause of this problem may be that some users have a subscription
 
    Thanks to Danny Herrlitz for this input!
 
-=======================================================================
-Q. How to migrate from Zimbra with an admin account?
-
-R. Use:
-
-  imapsync ... --user1 "normal_user" --authuser1 "admin_user"  --password1 "admin_user_password"
-
-To setup or use a Zimbra admin user see:
-https://zimbra.github.io/adminguide/8.8.9/index.html#_administrator_accounts
-
-Thanks to Richard Street from thinkround for this tip.
-
 =======================================================================
 =======================================================================

FAQ.d/htaccess.txt

@@ -1,5 +1,5 @@
 
-# $Id: htaccess.txt,v 1.27 2019/12/08 16:05:27 gilles Exp gilles $
+# $Id: htaccess.txt,v 1.29 2020/10/01 10:40:23 gilles Exp gilles $
 
 
 AddDescription "<b>Back</b> to Imapsync main page."                     ..
@@ -15,6 +15,7 @@ AddDescription "<b>Connection issues</b>."                              FAQ.Conn
 AddDescription "<b>Contacts & Calendars & Chat issues</b>."             FAQ.Contacts_Calendars.txt
 AddDescription "<b>Dates issues</b>."                                   FAQ.Dates.txt
 AddDescription "<b>David Tobit</b> accounts."                           FAQ.David_Tobit.txt
+AddDescription "<b>Docker</b> usage."                                   FAQ.Docker.txt
 AddDescription "<b>Domino</b> accounts."                                FAQ.Domino.txt
 AddDescription "<b>Dovecot</b> accounts."                               FAQ.Dovecot.txt
 AddDescription "<b>Duplicated</b> messages issues."                     FAQ.Duplicates.txt
@@ -36,6 +37,7 @@ AddDescription "<b>Migration Plan</b>."                                 FAQ.Migr
 AddDescription "<b>The Online UI service Q&R</b>."                      FAQ.OnlineUI.txt
 AddDescription "<b>Password & special characters on Windows</b>."       FAQ.Passwords_on_Windows.txt
 AddDescription "<b>Password & special characters on Unix</b>."          FAQ.Passwords_on_Unix.txt
+AddDescription "<b>Password & special characters on Mac OS X</b>."      FAQ.Passwords_on_Mac.txt
 AddDescription "<b>POP3 issues</b>."                                    FAQ.POP3.txt
 AddDescription "<b>Principles & design decisions</b>."                  FAQ.Principles.txt
 AddDescription "<b>Selecting messages</b>."                             FAQ.Messages_Selection.txt
@@ -50,6 +52,7 @@ AddDescription "<b>SSL errors</b>."                                     FAQ.SSL_
 AddDescription "<b>SmarterMail</b> accounts."                           FAQ.SmarterMail.txt
 
 AddDescription "<b>TTL (Time To Live) delay in DNS configuration</b>."  FAQ.TTL.txt
+AddDescription "<b>Why Imapsync can't do two ways syncs</b>."           FAQ.Two_Ways_Sync.txt
 AddDescription "<b>Option --usecache and inodes</b>."                   FAQ.Use_cache.txt
 AddDescription "<b>User concurrent access</b>."                         FAQ.User_Concurrent_Access.txt
 AddDescription "<b>Virus scanners on Imapsync</b>."                     FAQ.Virus.txt