Mirrors/imapsync
1
0
Fork 0
mirror of https://github.com/imapsync/imapsync.git synced 2024-11-16 15:52:47 +01:00
Code Issues Releases Activity

Update FAQ.d

Browse Source
This commit is contained in:
Florent Hemmi 2021-05-01 17:46:04 +02:00
parent 8d2730ac65
commit e7fecaa480
No known key found for this signature in database
GPG Key ID: 6F3D7079D4E2418D
46 changed files with 2768 additions and 864 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

67
FAQ.d/FAQ.APPEND_errors.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.APPEND_errors.txt,v 1.6 2019/02/16 22:38:49 gilles Exp gilles $
$Id: FAQ.APPEND_errors.txt,v 1.12 2021/02/10 19:42:42 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -11,15 +11,71 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.APPEND_errors.txt
Questions answered in this FAQ are:
Q. I have an "could not append" error with "Message contains invalid header" at
the end, like:
Err 1/20: - msg INBOX/6 {40666} could not append ... NO Message contains invalid header
What can I do to transfer these emails?
Q. For some messages, the imapsync log says
"could not append", sometimes followed by an explicit message
describing what went wrong, or sometimes followed by a not very
useful message "socket closed while reading data from server"
What can I do?
R0. Well, the problem is that the "socket closed ..." error message happens
in several different issues. So I list here several potential issues
Now the questions again with their answers.
=======================================================================
Q. I have an "could not append" error with "Message contains invalid header"
at the end, like:
Err 1/20: - msg INBOX/6 {40666} could not append ... NO Message contains invalid header
What can I do to transfer these emails?
R0. Append in the IMAP protocol is the command to add a message in the folder.
The error string "NO. Message contains invalid header" comes from the
destination IMAP server at host2, it doesn't like the message and rejects it.
R1. Header lines must be in 7bit encoding.
If they aren't in 7bit the IMAP server can refuse them as they break
the IMAP RFC specification.
A solution is to transform all 8bit characters to the X character with:
imapsync ... --regexmess 'tr [\x80-\xff] [X]'
Caveat: this transformation transforms both the email header and its body.
It's not perfect but I guess it's better than no message transferred.
R2. Some crappy email systems, like virus checkers or any other crappy
software tool dealing with your messages, can add headers to tell
the world that they've done a great job. But sometimes they didn't,
and they fucked up your messages by not respecting the standard
and added ugly non-rfc compliant headers.
To remove an ugly header, let's call it "X-Spam-Report", that spreads over
several lines beginning or, that's the ugly part, not beginning with a space:
imapsync ... --regexmess 's{X-Spam-Report:.*?\n(^[a-zA-Z0-9\-]+:|^\r?\n)}{$1}xms'
Thanks to Damien SAUTEREAU for reporting and solving this issue.
=======================================================================
Q. For some messages, the imapsync log says
"could not append", sometimes followed by an explicit message
describing what went wrong, or sometimes followed by a not very
useful message "socket closed while reading data from server"
What can I do?
R0. Append errors are host2 problems, append is the imap term to copy
a message to the destination account.
A problem is that the assosiated "socket closed ..." error message happens
for several different issues. So I list here several potential issues
and their solutions if they exist.
R1 deals about too long lines in messages on Windows.
R2 deals about too long lines in messages on Unix.
R3 deals with quota reached.
R1. On Windows, add --regexmess "s,(.{9900}),$1\r\n,g"
@ -77,5 +133,8 @@ OPTIONS
contains eight-bit characters to quoted-printable encoding.
R2.
R3. Look for the quota limit on host2. If it is the current mailbox size, don't
search elsewhere. Increase the quota on the destination account.
=======================================================================
=======================================================================

14
FAQ.d/FAQ.Admin_Authentication.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Admin_Authentication.txt,v 1.10 2019/10/31 23:13:50 gilles Exp gilles $
$Id: FAQ.Admin_Authentication.txt,v 1.12 2021/02/19 13:44:18 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -19,16 +19,16 @@ server and its configuration.
You have to follow this road:
* Determine what is the imap software server
* Find out wether it supports an admin account or not
* Find out whether it supports an admin account or not
* If no, leave this reading, take an handkerchief and cry
* If yes, search how the imap software server supports
an admin account (see below)
* Figure out wether an admin account is actually configured or not,
* If not then configure it,
* Figure out whether an admin account is actually configured or not,
* If not then configure it. If you can't configure it: handkerchief.
* Finally, check an imapsync authentication via this admin account for
a standard user, option --justlogin is nice for doing this.
a standard user, the option --justlogin is nice for doing this.
* Use it for all users.
* Next, read their email messages, enter their personnal life etc.
* Next, read their email messages, enter their personal life etc.
I'm kidding.
Good luck!
@ -41,8 +41,8 @@ Known imap software servers supporting imap admin authentication:
* Dovecot. See the file FAQ.Dovecot.txt
* Zimbra. See the file FAQ.Zimbra.txt
* Kerio See the file FAQ.Kerio.txt
* Cyrus-imap. See the file FAQ.Cyrus.txt
* UW-imap. See the file FAQ.General.txt
* Cyrus-imap. See the file FAQ.General.txt
* Sun Java Enterprise System/SunOne/iPlanet. See the file FAQ.General.txt
Known imap software servers NOT supporting imap admin authentication:

31
FAQ.d/FAQ.Archiving.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Archiving.txt,v 1.18 2019/10/23 17:56:28 gilles Exp gilles $
$Id: FAQ.Archiving.txt,v 1.24 2021/04/18 09:11:32 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -30,6 +30,9 @@ Q. How to archive a whole year in a subfolder?
Q. How to archive a whole year in subfolders but without subsubfolders?
Now the questions again with their answers.
https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
=======================================================================
Q. How can I archive different accounts on the same destination account,
@ -83,6 +86,10 @@ Examples:
imapsync ... --user2 Bar --subfolder1 BarArchiv
Read the previous FAQ item "How can I archive different accounts on the same
destination account, each account on a separate folder?"
=======================================================================
Q. How to move emails from one IMAP folder to another either on the
same IMAP server or a different one?
@ -121,23 +128,37 @@ R2. Imapsync can't but rick-sanders-imap-tools can!
is more dangerous than doing no backups at all. No backups makes
people act in a safer way, well, usually...
R3. Use ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos.
R3. Use isync or mbsync
https://isync.sourceforge.io/
R4. Use ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos.
ImportExportTools does not do IMAP as a destination.
R5. There is a free online service (French language) allowing you to do
a Maildir or EML or HTML backup at
https://www.powermail.fr/sauvegarder-une-boite-mail/
Prefer Maildir or EML format if you want to be able to restore the
archive. HTML format will be ok to only view the messages, not
restore them to so imap server account.
I have never used any of this tools so asking me for support with them
won't go far away.
=======================================================================
Q. How to archive a whole year in a subfolder?
R.
R. Example:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
--folder2 ARCHIVE_2018 --justfolders --dry
--folder2 ARCHIVE_2018
=======================================================================
Q. How to archive a whole year in subfolders but without subsubfolders?
R. Example:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
--sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}' --justfolders --dry
--sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}'
=======================================================================
=======================================================================

52
FAQ.d/FAQ.Authentication_failure.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Authentication_failure.txt,v 1.9 2019/01/11 13:13:08 gilles Exp gilles $
$Id: FAQ.Authentication_failure.txt,v 1.13 2021/03/02 19:24:22 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -17,6 +17,8 @@ Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2
Q. Imapsync fails with the following error, what can I do?
Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom
Now the questions again with their answers.
=======================================================================
Q. Imapsync fails with the following error, what can I do?
@ -55,9 +57,11 @@ Here are some advices to get you pass this difficult stage of authentication:
* It is sometimes very hard to quote correctly unusual characters,
especially on Windows. See
especially on Windows (and Mac sometimes). See
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
The quickest trick may be to change the password temporally
with easy characters like the classical alphabet, a long
string still ensure strong security.
@ -69,20 +73,54 @@ Here are some advices to get you pass this difficult stage of authentication:
* Try --authmech1 CRAM-MD5 (or --authmech2 CRAM-MD5)
* Try --authmech1 PLAIN (or --authmech2 PLAIN)
* If you want to play manually the IMAP protocol on your server,
here is an example with the command telnet, user test1
and password secret1 on the host test.lamiral.info.
* Manually test a login via ssl.
Use either ncat or telnet-ssl or openssl commands like in the
following examples with imap.gmail.com server:
telnet test.lamiral.info 143
ncat --ssl -C imap.gmail.com 993
telnet-ssl -z ssl imap.gmail.com 993
openssl s_client -crlf -connect imap.gmail.com:993
Replace imap.gmail.com by your imap server.
The three commands do the same work here.
Tipical dialog for an imap LOGIN command:
* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
a LOGIN "gilles.lamiral@gmail.com" "secret"
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
b LOGOUT
* BYE LOGOUT Requested
b OK 73 good day (Success)
The client part you have to type is
a LOGIN ...
b LOGOUT
while replacing ... by your credentials values,
other lines are the server responses. Letters
a and b are identifiers of IMAP commands, you have to
type them but you can chose any string instead of a or b.
Some other examples:
ncat --ssl test.lamiral.info 993
c1 LOGIN test1 "secret1"
c2 LOGOUT
Same example using an ssl conneection:
openssl s_client -crlf -connect test.lamiral.info:993
c1 LOGIN test1 "secret1"
c2 LOGOUT
ncat --ssl -C outlook.office365.com 993
a LOGIN gilles.lamiral@outlook.com topsecret
b LOGOUT
telnet-ssl -z ssl test.lamiral.info 993
c1 LOGIN test1 "secret1"
c2 LOGOUT
=======================================================================
Q. Imapsync fails with the following error, what can I do?
Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom

46
FAQ.d/FAQ.Bandwidth.txt Normal file
View File

@ -0,0 +1,46 @@
$Id: FAQ.Bandwidth.txt,v 1.2 2021/04/19 11:46:47 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Bandwidth.txt
=======================================================================
Imapsync bandwidth used
=======================================================================
Questions answered in this FAQ are:
Q. What is the bandwidth used by imapsync?
Now the questions again with their answers.
=======================================================================
Q. What is the bandwidth used by imapsync?
R. From the host where imapsync runs, it opens two imap connections,
one with the source account at host1, one with the destination
account at host2.
So, the global bandwidth used is twice the volume of the source
account, one volume to download the messages from host1,
one volume to upload those messages to host2.
If the host2 is already filled with the messages, imapsync doesn't
transfer them and then the volume transferred is small, this volume
is just the IMAP commands needed to identify the messages on both sides.
There is no local cache of the email messages, except when the
message is very big, it is temporarily saved locally.
The biggest message seen so far on online /X is 3.08 GiB
while the biggest message transferred is 1.51 GiB so
I suspect a bug here. Drop me a note if you encounter the same
issue, I'll then dig into it, ie, I'll create a 2 or 3 GiB message
and play with it :-)
=======================================================================
=======================================================================

35
FAQ.d/FAQ.Big_Mailbox.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Big_Mailbox.txt,v 1.4 2019/07/31 10:37:15 gilles Exp gilles $
$Id: FAQ.Big_Mailbox.txt,v 1.7 2021/02/01 21:47:41 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -21,18 +21,20 @@ Q. How to deal with huge mailboxes, whose size is over dozens of GB?
R1. It should be ok with imapsync.
In case imapsync seems to stall when sizing the folders or before
syncing a huge folder and you wander wether it is doing something or
just frozen, you can add option --debugimap. Option --debugimap shows
what is currently done, it's quite a big output but it helps waiting,
saying to ourselves "ok it's long but it's working, let's wait a
little more". The real purpose of --debugimap is to show genuine IMAP
commands used and their responses. The problem with --debugimap is
that it is very verbose and so with a big mailbox it generates a very
BIG logfile.
Sometimes with big folders, imapsync seems to stall when sizing the
folders or maybe before syncing a huge folder. So you wonder whether
imapsync is doing something or being just frozen. A solution to
increase the verbosity is to add the option --debugimap. Option
--debugimap shows what is currently done, it's quite a big output but
it helps to wait, saying to ourselves "ok it's long but it's working,
let's wait a little more". The real purpose of --debugimap is to show
genuine IMAP commands used and their responses. The problem with
--debugimap is that it is very verbose and so with a big mailbox it
generates a very BIG logfile.
R2. If it is not ok with imapsync then consider spliting the whole
sync into smaller chunks using --search
R2. If it is not ok with imapsync, the sync finally fails, then
consider spliting the whole sync into smaller chunks using
the option --search
For example, select messages only for a given year:
@ -41,5 +43,14 @@ For example, select messages only for a given year:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018"
imapsync ... --search "SENTSINCE 1-Jan-2019 SENTBEFORE 31-Dec-2019"
R3. Use option --useuid
Option --useuid speeds up the process but --useuid implies --usecache
and --usecache may trigger some issues sometimes. Read:
https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
Don't use --useuid in Docker context, the cache will be destroyed
each time, or put it in a stable place, I don't know how to do it,
I just know it's possible and not very difficult.
=====================================================================
=====================================================================

76
FAQ.d/FAQ.Contacts_Calendars.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Contacts_Calendars.txt,v 1.12 2019/04/30 13:04:18 gilles Exp gilles $
$Id: FAQ.Contacts_Calendars.txt,v 1.14 2020/11/02 16:04:09 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -29,10 +29,10 @@ R. No, Imapsync can't migrate Contacts and Calendars.
No way via IMAP, no way via imapsync.
So it's a good idea to avoid syncing contacts and calendars when
using imapsync. See below
using imapsync. See at the end of this document
"How can I avoid copying Calendar or Contacts folders?"
But see also the question at the end of this document
But see also the question below
"How can I copy or synchronize Calendars or Contacts?".
=======================================================================
@ -41,6 +41,45 @@ Q. Can I copy or sync Chat messages with imapsync?
R. No, Imapsync can't migrate Chat messages (those in GSuite for example).
The explanation is the same as with Calendars or Contacts.
=======================================================================
Q. How can I copy or synchronize Calendars or Contacts?
R1. Synchronizing Calendars or Contacts can't be done with imapsync.
For detailed explanations, see the question above
"Can I copy or sync Calendars or Contacts with imapsync?".
R2. Synchronizing Calendars or Contacts can be done, not
with imapsync, depending on the imap server software tool used.
a) From Exchange to Exchange, export contacts and calendar to
PST format files on host1 and import them on host2.
b) From Gmail to Gmail, export and import calendars in ical format,
extension for those files is .ics.
Contacts can be copied using a csv file. See the help page
http://support.google.com/mail/bin/topic.py?hl=en&topic=1669027
c) For Outlook, use caldavsynchronizer.
Homepage: http://caldavsynchronizer.org/
Outlook CalDav Synchronizer is a free Outlook Plugin, which synchronizes
events, tasks and contacts between Outlook and Google, SOGo, Nextcloud or
any other CalDAV or CardDAV server.
Supported Outlook versions are Office 365, 2019, 2016, 2013, 2010 and 2007.
Free and Open Source software tool.
d) Etc. Search the web. There's also specific tools and paid services.
There's no silver bullet to migrate Calendars and Contacts,
if you find one, tell me!
Also consider asking experts at Sumatra company
https://twitter.com/sumatra_dev
http://sumatra.com/
e) I plan to make tools for that but so far nothing has began
more than this discussion:
http://www.linux-france.org/prj/imapsync_list/msg01797.html
http://www.linux-france.org/prj/imapsync_list/msg01811.html
=======================================================================
Q. How can I avoid copying Calendar or Contacts folders?
@ -71,36 +110,5 @@ R. You can avoid synchronizing Calendar or Contacts folders with
imapsync ... --exclude "^Calendar\$" --exclude "^Contacts\$"
=======================================================================
Q. How can I copy or synchronize Calendars or Contacts?
R1. Synchronizing Calendars or Contacts can't be done with imapsync.
For detailed explanations, see the question above
"Can I copy or sync Calendars or Contacts with imapsync?".
R2. Synchronizing Calendars or Contacts can be done, not
with imapsync, depending on the imap server software tool used.
a) From Exchange to Exchange, export contacts and calendar to
PST format files on host1 and import them on host2.
b) From Gmail to Gmail, export and import calendars in ical format,
extension for those files is .ics.
Contacts can be copied using a csv file. See the help page
http://support.google.com/mail/bin/topic.py?hl=en&topic=1669027
c) Etc. Search the web. There's also specific tools and paid services.
There's no silver bullet to migrate Calendars and Contacts,
if you find one, tell me!
Also consider asking experts at Sumatra company
https://twitter.com/sumatra_dev
http://sumatra.com/
d) I plan to make tools for that but so far nothing has began
more than this discussion:
http://www.linux-france.org/prj/imapsync_list/msg01797.html
http://www.linux-france.org/prj/imapsync_list/msg01811.html
=======================================================================
=======================================================================

127
FAQ.d/FAQ.Cyrus.txt Normal file
View File

@ -0,0 +1,127 @@
#!/bin/cat
$Id: FAQ.Cyrus.txt,v 1.2 2021/02/19 13:39:03 gilles Exp gilles $
=======================================================================
Imapsync tips for the Cyrus imap server
=======================================================================
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
Questions answered in this FAQ are:
Q. How to migrate from cyrus with an admin account?
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail but the tool copies everything in /home/user, how
can i avoid that?
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail directory. When imapsync creates the folders in
the new cyrus imap server, it makes a folder "mail" and below that
folder puts all the mail folders the user have in /home/user/mail,
i would like to have all those folders directly under INBOX.
Q. Migrating from Groupwise to Cyrus
Now the questions again with their answers.
======================================================================
Q. How to migrate from cyrus with an admin account?
R. Use:
imapsync ... \
--authuser1 admin_user ----password1 admin_user_password \
--user1 foo_user --ssl1
Instead of --ssl1 the alternative --tls1 can be used.
With --authuser1, the option --authmech1 PLAIN is set
automatically, you don't have to add it.
PLAIN authentication is the only way to go with --authuser1 for now.
So don't use --authmech1 SOMETHING with --authuser1 admin_user,
it will not work.
Same behavior with the --authuser2 option.
Do not forget the option --ssl1 or --tls1 since PLAIN auth is only
supported with ssl encryption most of the time. But it can
work without --ssl1 nor --tls1 if PLAIN is permitted in clear text
transmissions (the normal mode).
Add the AdminAccount to admins line in /etc/imapd.conf
Give AdminAccount lrswipkxtecda to the Cyrus Imap account
being migrated from, "joe" here.
Here is an example:
imapsync \
--host1 server1 \
--user1 joe \
--authuser1 AdminAccount \
--password1 AdminAccountPassword \
--ssl1 \
--host2 server2 \
--user2 joe \
--password2 joespassonserver2 \
--exclude "^user\."
=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail but the tool copies everything in /home/user, how
can i avoid that?
Two solutions:
R. Use
imapsync ... --include '^mail'
R. or (better)
imapsync ... --subscribed --subscribe
=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail directory. When imapsync creates the folders in
the new cyrus imap server, it makes a folder "mail" and below that
folder puts all the mail folders the user have in /home/user/mail,
i would like to have all those folders directly under INBOX.
R. Use
imapsync ... --regextrans2 's/^mail/INBOX/' --dry
look at the simulation and if all transformations seem
good then remove the --dry option.
=======================================================================
Q. Migrating from Groupwise to Cyrus
R. By Jamie Neil:
I eventually managed to get the mail to migrate without errors using the
following options:
--maxage 3650
- some messages just don't seem to want to transfer and produce the
perl errors I mentioned before. This prevents the errors, but the
bad messages don't transfer.
Even though the mail migrated OK, there are a couple of gotchas with
Groupwise IMAP:
1) Some of the GW folders are not real folders and are not available
to IMAP, the main problem one being "Sent Items". I could find no way
of coping the contents of these folders. The nearest I got was to
create a "real" folder and copy/move the sent items into it, but
imapsync still didn't see the messages (I think because there is
something funny about the reported dates/sizes).
It think this problem has been rectified in GW6.5.
2) Version 6.0.1 of the Groupwise Internet Connector sucks. I was
getting server aborts when I pushed it a bit hard! I eventually had to
upgrade to 6.0.4 which seems to be a lot more stable.
=======================================================================

22
FAQ.d/FAQ.DBmail.txt Normal file
View File

@ -0,0 +1,22 @@
#!/bin/cat
$Id: FAQ.DBmail.txt,v 1.1 2020/11/04 08:43:05 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.DBmail.txt
=======================================================================
Imapsync tips for DBmail accounts.
=======================================================================
=======================================================================
Q. How to migrate from or to DBmail?
R. In order to avoid duplicates, use:
imapsync ... --useheader Message-Id
More explanations and details at
https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt

58
FAQ.d/FAQ.Docker.txt
View File

@ -1,50 +1,86 @@
#!/bin/cat
# $Id: FAQ.Docker.txt,v 1.6 2019/12/11 15:39:14 gilles Exp gilles $
# $Id: FAQ.Docker.txt,v 1.7 2020/01/08 05:02:55 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt
=======================================================================
===========================================================================
Installing and using imapsync docker image
=======================================================================
===========================================================================
Questions answered in this FAQ are:
Q. How can I install and use the imapsync Docker image on my system?
Q. How can I install and use the imapsync Docker image on my amd64 system?
Q. How can I install and use the imapsync Docker image on my i386 system?
Q. How can I install or update only the imapsync Docker image on my system?
Q. Where is the imapsync Docker hub location?
Q. How can I mask the passwords on the command line without mounting
inside the container?
Q. Any tips for the Docker Mailcow distribution?
Now the questions again with their answers.
=======================================================================
Q. How can I install and use the imapsync Docker image on my system?
===========================================================================
Q. How can I install and use the imapsync Docker image on my amd64 system?
R. Install Docker on your system. Once Docker is installed on your system,
all you have to do in order to run imapsync is the command line:
docker run gilleslamiral/imapsync imapsync <usual imapsync arguments>
=======================================================================
===========================================================================
Q. How can I install and use the imapsync Docker image on my i386 system?
R. You can't for now. It will throw this error when running it:
docker run gilleslamiral/imapsync imapsync
standard_init_linux.go:211: exec user process caused "exec format error"
I plan to make the image multi-plateform. Drop me a note if you are
eager to have it.
===========================================================================
Q. How can I install or update only the imapsync Docker image on my system?
R. To install or update the imapsync image, run:
docker pull gilleslamiral/imapsync
=======================================================================
===========================================================================
Q. Where is the imapsync Docker hub location?
R. Here:
https://hub.docker.com/r/gilleslamiral/imapsync/
=======================================================================
===========================================================================
Q. How can I mask the passwords on the command line without mounting
anything inside the container?
R. Use a file defining the environment variables
IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2
cat ./secret.txt
IMAPSYNC_PASSWORD1=secret1
IMAPSYNC_PASSWORD2=secret2
Use that file like this for a run:
docker run --env-file ./secret.txt gilleslamiral/imapsync imapsync ...
See:
https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
https://stackoverflow.com/a/30494145/491175
===========================================================================
Q. Any tips for the Docker Mailcow distribution?
R. With the Mailcow distribution, imapsync is running inside a Docker
@ -61,5 +97,5 @@ See the original Mathilde's comment:
https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077
=======================================================================
=======================================================================
===========================================================================
===========================================================================

16
FAQ.d/FAQ.Domino.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: FAQ.Domino.txt,v 1.12 2019/10/31 23:13:31 gilles Exp gilles $
# $Id: FAQ.Domino.txt,v 1.13 2020/09/06 20:33:46 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -103,8 +103,9 @@ For Domino 853FP6 on Linux, we used this command on Unix:
--prefix2 "" --sep2 "\/" --messageidnodomain \
--regexmess 's{\A(.*?(?! ^$))^Date:(.*?)$}{$1Migratedbyus:$2\nx-MailDate:$2}gxms'
=======================================================================
Q. Whan I use --maxage imapsync sees no messages at all.
Q. When I use --maxage option, imapsync sees no messages at all.
What can I do?
R. Use --noabletosearch
@ -113,6 +114,17 @@ R. Use --noabletosearch
Same thing with --minage
=======================================================================
Q. On Domino this errors occurs:
"Login failure, cannot convert Mailbox: code 02:46"
R. Follow
https://atnotes.de/index.php?topic=44888.0
Use:
load convert -e mail\mailfilename.nsf
=======================================================================
Q. My usual email software client like Thunderbird sees more messages
than imapsync. How can it be possible?

38
FAQ.d/FAQ.Dovecot.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Dovecot.txt,v 1.7 2019/09/25 09:25:52 gilles Exp gilles $
$Id: FAQ.Dovecot.txt,v 1.12 2021/04/30 13:02:06 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -18,8 +18,17 @@ Q. Subfolders are not created by imapsync and the error is
Q. How to migrate to Dovecot with an admin/MasterUser account?
Q. I have a mailbox that "dovecot backup" gives errors on that
I can't get rid of: "UID inserted in the middle of mailbox".
What can I do?
Q. Imapsync encounters timeouts and Dovecot log says
"mailpiler dovecot: imap(xxx): Error: Timeout (180s) while waiting for dotlock for transaction log file ..."
What can I do?
Now the questions again with their answers.
=======================================================================
Q. How to migrate to Dovecot with an admin/MasterUser account?
@ -27,8 +36,22 @@ R. Dovecot uses the same syntax as uw-imap
imapsync ... --user2="loginuser*admin_user" --password2 "admin_user_password"
To setup a Dovecot MasterUser see
http://wiki2.dovecot.org/Authentication/MasterUsers
To setup a Dovecot MasterUser follow the document
https://doc.dovecot.org/configuration_manual/authentication/master_users/
=======================================================================
Q. I have a mailbox that "dovecot backup" gives errors on that
I can't get rid of: "UID inserted in the middle of mailbox".
What can I do?
Q. Solution, run:
doveadm -v deduplicate -m -u martin mailbox A/Posted
it will get rid of the duplicates (I can also be run without "-m")
and this also allows "dovecot backup" to work.
Thanks to Martin Ward for this input!
=======================================================================
Q. Subfolders are not created by imapsync and the error is
@ -47,6 +70,15 @@ https://superuser.com/questions/813362/sub-folders-in-dovecot
https://wiki2.dovecot.org/MailLocation/Maildir
=======================================================================
Q. Imapsync encounters timeouts and Dovecot log says
"mailpiler dovecot: imap(xxx): Error: Timeout (180s) while waiting for dotlock for transaction log file ..."
What can I do?
R. It looks like an NFS issue with Dovecot.
Solve it with Dovecot support or use something else than NFS.
I'll be glad to share here any solution you find.
(Thanks to Mate Skoblar for reporting this issue)
=======================================================================
=======================================================================

71
FAQ.d/FAQ.Duplicates.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Duplicates.txt,v 1.20 2019/05/17 10:10:30 gilles Exp gilles $
$Id: FAQ.Duplicates.txt,v 1.23 2021/04/22 13:28:41 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -9,6 +9,38 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
Imapsync tips about duplicated messages issues.
=======================================================================
Questions answered in this FAQ are:
Q. How does imapsync identify messages and duplicates?
Q. How can I know if imapsync will generate duplicates on a second run?
Q: I found multiple copies, duplicates, when I run imapsync twice or
more. What is hell is happening?
Q. imapsync calculates 479 messages in a folder but only transfers 400
messages. What is happening?
Q. imapsync doesn't synchronize duplicates by default but I want to.
How can I synchronize duplicates?
Q. How can I remove duplicates in a lone account?
Now the questions again with their answers.
=======================================================================
Q. How does imapsync identify messages and duplicates?
R. Imapsync by default identify messages by their headers "Message-Id"
and "Received". Usually, for a given message, "Message-Id" appears one
time while multiple "Received" headers are common.
For imapsync, messages with the same "Message-Id" and "Received" headers
are consider identical, ie, duplicates.
=======================================================================
Q. How can I know if imapsync will generate duplicates on a second run?
@ -32,7 +64,8 @@ Messages transferred : 0
Messages skipped : 1555
=======================================================================
Q: Multiple copies, duplicates, when I run imapsync twice ore more.
Q: I found multiple copies, duplicates, when I run imapsync twice or
more. What is hell is happening?
R0. First, some explanations to understand the issue.
Normally and by default, imapsync doesn't generate duplicates.
@ -172,7 +205,7 @@ trick to keep the --useheader filtering behavior is to use
=======================================================================
Q. imapsync calculates 479 messages in a folder but only transfers 400
messages. What's happen?
messages. What is happening?
R1. Unless --useuid is used, imapsync considers a header part
of a message to identify a message on both sides.
@ -190,6 +223,38 @@ R2. With option --useuid imapsync doesn't use headers to identify
messages on both sides but it uses their imap uid identifier.
In that case duplicates on host1 are also transferred on host2.
=======================================================================
Q. imapsync doesn't synchronize duplicates by default but I want to.
How can I synchronize duplicates?
R. Use the option --useuid
If you have already synchronized two mailboxes without --useuid then
using it right away will generate duplicates on host2. To avoid that
behavior, you have to perform a first run with --usecache to build
the local UID cache. Then the next runs with --useuid
There are potentially issues with --usecache. They can be solved.
Read the document FAQ.Use_cache.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
So to finalise how to synchronize duplicates:
imapsync ... --tmpdir . --usecache
imapsync ... --tmpdir . --useuid
imapsync ... --tmpdir . --useuid
...
Here --tmpdir value is the dot "." meaning "current directory".
Surrounding it with double-quotes is optional.
If the two mailboxes haven't been already synchronized then the
first run with --usecache is useless.
I shall add an option --syncduplicates that syncs duplicates
without any cache needed and make it the default but it's not
coded yet.
=======================================================================
Q. How can I remove duplicates in a lone account?

60
FAQ.d/FAQ.Emptying.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Emptying.txt,v 1.8 2019/04/30 12:52:10 gilles Exp gilles $
$Id: FAQ.Emptying.txt,v 1.11 2020/10/13 16:24:43 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -10,6 +10,15 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Emptying.txt
Imapsync tip to empty an account.
=======================================================================
Questions answered in this FAQ are:
Q. How to delete all emails of all folders of an account with imapsync?
Q. How to delete all folders of an account with imapsync?
Q. How to delete some folders of an account with imapsync?
Q. How to revert a sync that went to the wrong account?
Now the questions again with their answers.
=======================================================================
Q. How to delete all emails of all folders of an account with imapsync?
@ -20,7 +29,7 @@ Option --noexpungeaftereach is just to speed up the deletions.
Example (real example):
./imapsync \
imapsync \
--host1 test.lamiral.info --user1 empty --password1 secret \
--host2 test.lamiral.info --user2 empty --password2 secret \
--delete1 --noexpungeaftereach
@ -31,10 +40,55 @@ Q. How to delete all folders of an account with imapsync?
R. Use the same account on both sides with the options
--delete1 --noexpungeaftereach --delete1emptyfolders
./imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders
imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders
It won't delete the folder INBOX since INBOX is mandatory in imap.
=======================================================================
Q. How to delete some folders of an account with imapsync?
R. Use the same account on both sides with the options
--delete1 --noexpungeaftereach --delete1emptyfolders
like when deleting all folders BUT select only the folders
you want to delete by using options to select folders,
like --folder or --folderrec or --include or --exclude
Example, to delete the folder Blabla
imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders --folder Blabla
To delete the four folders Kalender Kontakte Aufgaben Notizen:
imapsync ... --include 'Kalender|Kontakte|Aufgaben|Notizen' \
--delete1 --noexpungeaftereach --delete1emptyfolders
I strongly recommend to use --dry first!
To get an overview of how to select folders, see
https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
=======================================================================
Q. How to revert a sync that went to the wrong account?
R. If all the wrong emails have some common characteristic that is not
shared by the good one, then you can search for them and delete them
using the same account as source and destination. For example, if all
the messages have the same email destination address as
"john_smith@example.com", then you can remove them with the following
command:
imapsync ... --search "TO john_smith@example.com" --delete1
Be very careful with deletions! Since this section deals with something
that was made wrong, you're in good place to make a new one!
There is many possibilities to select messages with different criterium.
See some of them by reading the document
https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
If the wrongly emails messages synced have no special common
characteristic then take an handkerchief and cry.
=======================================================================
=======================================================================

315
FAQ.d/FAQ.Exchange.txt
View File

@ -1,55 +1,46 @@
$Id: FAQ.Exchange.txt,v 1.57 2019/12/04 18:26:28 gilles Exp gilles $
$Id: FAQ.Exchange.txt,v 1.74 2021/03/23 14:53:33 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
=======================================================================
Imapsync tips for Exchange 20xx and Office365.
Imapsync tips for Exchange 2000 2003 2007 2010 2013 2016 2019
=======================================================================
Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
Q. Can I use imapsync to transfer from or to Exchange accounts?
Q. How to sync from XXX to Exchange 2010/2013/2016
Q. How to sync from Office365 to XXX?
Q. How to sync from XXX to Office365
Q. For Office365 I have double and triple checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
Q. How to sync from XXX to Exchange 2010/2013/2016/2019
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that? Happens with Office365 or Exchange 2016.
What can I do with that?
Q. Office365 throttles the sync and says:
Q. Exchange throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
Q. Office365 users complain that a folder named "Files" contains
messages with no sender.
Q. For Exchange I have double and triple checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
Q. Exchange fails with "User is authenticated but not connected".
Q. Exchange fails with "BAD Command received in Invalid state".
Q. From XXX to Exchange 2013 or Office365, read receipts are all
resent again after a sync. Even for old messages. How can I fix that?
Q. From XXX to Exchange 2013, read receipts are all resent again
after a sync. Even for old messages. How can I fix that?
Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
Q. From XXX to Exchange 2010/2013 I get this error message
sometimes: "BAD Command Argument Error 11". What does it mean?
Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does
Q. From XXX to Exchange 2010 or 2013, the flag Flagged does
not seem to be well synced. What can I do?
Q. How to migrate from or to Exchange 2007/2010/2013 with an
@ -57,10 +48,7 @@ Q. How to migrate from or to Exchange 2007/2010/2013 with an
Q. How to migrate from or to Exchange 2016 with an admin/authuser account?
Q. How to migrate from or to Office 365 with an admin/authuser account?
Q. How to migrate from or to Exchange 2003 with an admin/authuser
account?
Q. How to migrate from or to Exchange 2003 with an admin/authuser account?
Q. Couldn't create folder [trash] "Mailbox already exists".
@ -78,20 +66,21 @@ Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server?
Now the questions again with their answers.
=======================================================================
Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
Q. Can I use imapsync to transfer from or to Exchange accounts?
R. Yes. But IMAP access to a Exchange or Office365 account is not always
R. Yes. But IMAP access to an Exchange account is not always
allowed by default so it has to be allowed in the server configuration
part.
Exchange: https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help
Office365: https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
=======================================================================
Q. How to sync from XXX to Exchange 2010/2013/2016
Q. How to sync from XXX to Exchange 2010/2013/2016/2019
R0. IMAP is not enable by default on Exchange, see how to enable it:
https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help
Also read
http://clintboessen.blogspot.com/2018/03/binding-certificate-breaks-imap-or-pop.html
R1. Following is a command line resume that solves most encountered
@ -135,73 +124,66 @@ that can be used like:
To get reformime, install the "maildrop" package.
=======================================================================
Q. How to sync from Office365 to XXX?
On Linux again, there is a good Python script in the tarball that can
fix several things that Exchange or O365 have issues with.
R. On Windows, use:
Use it like this:
imapsync.exe ... --office1
./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
On Unix, use:
imapsync ... --office1
Option --office1 is like (release 1.970):
imapsync ... --host1 outlook.office365.com \
--ssl1 \
--exclude "^Files$"
It often does some miracles on messages.
=======================================================================
Q. How to sync from XXX to Office365
R0. IMAP is not enable by default on Office365, see how to enable it:
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
https://docs.microsoft.com/en-us/exchange/allow-pop3-imap4-and-smtp-server-settings-to-be-viewed-by-end-users-in-outlook-web-app-exchange-2013-help
https://support.context.io/hc/en-us/articles/213586243-Enabling-IMAP-for-Office365
R. Here is a command line resume that solves most encountered issues when
migrating to Office365. It's similar with Exchange except for some
values. To understand or change the details you have to read
next Q/R sections.
imapsync ... --office2
which is equivalent to (in imapsync release 1.870):
imapsync ... \
--host2 outlook.office365.com \
--ssl2 \
--maxsize 45000000 \
--maxmessagespersecond 4 \
--disarmreadreceipts \
--regexmess "s,(.{10239}),$1\r\n,g" \
--f1f2 "Files=Files_renamed_by_imapsync"
On Linux, you can also try the "reformime" command
that can be used like:
imapsync ... --maxlinelengthcmd "reformime -r7"
To get reformime, install the "maildrop" package.
=======================================================================
Q. For Office365 I have double and triple checked the username and
Q. For Exchange I have double and triple checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
R1. Triple check the hostname then. Try all of these:
* imap-mail.outlook.com
* imap.outlook.com
* outlook.office365.com
R2. Also triple check a license is assigned to that account
in Office365.
R1. Disable double-step authentication, also known as 2-factor,
2-step authentication on the Azure/Active Directory portal.
R2. Microsoft introduced something called "security defaults" which is
enabled by default for new tenants. One of the rules blocks IMAP
access as of imapsync. The funny thing is that you can't disable
single rules of this security package without buying additional
licenses. Switching the whole thing off allows the login.
Also, disable double-step authentication on the Azure/Active Directory
portal.
See here:
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
Thanks to Stephan Buhre for this R2 answer.
R3. I want to migrate to Exchange 2019. I haven't been able to sync
using admin credentials even if I've tried and made all the
configuration mentioned in the FAQ.Exchange.
The error was always "LOGIN failed".
My problem was that I have been used an admin account which haven’t
attached a mailbox in Exchange. This is my admin account which I use
only for configuration in Exchange, not for emails.
So, after I’ve created in Exchange a mailbox for this user, the sync
started to work. I can confirm that imapsync is working for Exchange
2019, with plain authentication, and full permissions of admin account
(this account must have mailbox attached) applied to regular user
mailbox account.
It doesn't work with the --authuser2 parameter. The syntax that
works for me is:
imapsync ... --host2 192.168.xxx.yyy --user2 domain.local\admin.account\regular.user --password2 "xxxxxxx"
or
imapsync ...--host2 192.168.xxx.yyy --user2 admin.account@mycompany.com\regular.user --password2 "xxxxxxx"
Thanks to Ioan Nechita for this R3 answer.
R4. Are there special characters in the password?
https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
R3. Try with a classic email client like Thunderbird and the same
parameters.
=======================================================================
Q. Exchange fails with "User is authenticated but not connected".
@ -221,32 +203,16 @@ R2. Miguel Alameda reported understanding and solving this issue
=======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that? Happens with Office365 or Exchange 2016
What can I do with that?
R0. It happens with Office365 and Exchange 2016.
See R1 for Office365
See R2 for Exchange 2016
R1. With Exchange 2019
R1. Office365 supports send/receive max message sizes of up to 150MB
but you need to make changes in your tenant(s) to support it.
Change the recipient Receive/Send size:
The following PowerShell command will increase the message sizes that
can be sent/received. The trick in getting IMAPSync to work is to
apply these settings to the accounts performing the migration,
NOT the accounts associated with the target mailbox (assuming you're
using service accounts to perform transfers on behalf of users).
Set-mailbox -Identity user@contoso.com -MaxReceiveSize 100mb -MaxSendSize 100mb
Set-mailbox -Identity $UPN -MaxReceiveSize 150mb -MaxSendSize 150mb
Thanks to Pasternak Denis for this Exchange 2019 tip.
e.g.
Set-mailbox -Identity "migrationaccount@testtenant.onmicrosoft.com" -MaxReceiveSize 150mb -MaxSendSize 150mb
We're transferring data between Office 365 tenants so we set these
values on the migration acounts in the source and target tenants.
Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto
for this FAQ item.
R2. With Exchange 2016
@ -275,7 +241,8 @@ Please note that this change may be lost each time you install an
cumulative update. After the change, restart the services exchange
transport role, exchange information store and exchange imap4 Backend.
(Source: https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde)
Source in Deutch:
https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde
This change made our Exchange installation accept bigger e-mails
in order to remove the imapsync Exchange error message
@ -283,29 +250,19 @@ in order to remove the imapsync Exchange error message
=======================================================================
Q. Office365 throttles the sync and says:
Q. Exchange throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
R. Exchange and Office365 have throttle mechanisms to limit any huge
R0. Exchange have throttle mechanisms to limit any huge
usage. Sometimes imapsync transfers are too stressful for servers.
The message
"Request is throttled. Suggested Backoff Time: 299961 milliseconds"
comes from the imap Office365 server, imapsync just reports it
before being disconnected from it.
To solve the throttles issues from 0365, there are two solutions
at least:
R1. Call Microsoft Office365 and ask them to remove the limits on your
mailboxes. That's not a joke, they do it for 90 days usually,
sometimes only after you reach the second technician you call,
the first one usually been not enough competent to understand
what you're talking about (I would be glad to remove this bad
fact).
R2. Play with options --maxbytespersecond or --maxmessagespersecond
R1. Play with options --maxbytespersecond or --maxmessagespersecond
or --exitwhenover
imapsync ... --maxbytespersecond 100_000
@ -317,61 +274,17 @@ R2. Play with options --maxbytespersecond or --maxmessagespersecond
I don't know the upper value that avoid the default throttling from 0365
and I guess it changes over time.
R3. For Exchange, in case throttle appears anyway, fix them with:
R2. For Exchange, in case throttle appears anyway, fix them with:
https://docs.microsoft.com/en-us/exchange/change-user-throttling-settings-for-specific-users-exchange-2013-help
See also:
http://www.linux-france.org/prj/imapsync_list/msg02072.html
(Subject: Re: [imapsync] From Exchange to Dovecot. Speed
Date: Mon, 03 Nov 2014 00:14:59 +0100)
Sometimes restarting the Exchange server is needed to take
into account the change in the configuration.
=======================================================================
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
R. The folder Files is a standard folder in Office365. It should not
be synced in IMAP. See the next question.
=======================================================================
Q. Office365 users complain that a folder named "Files" contains
messages with no sender.
R0. To fix this, add --exclude Files
imapsync ... --exclude Files
If you use --office1 then imapsync will add this exclusion
automatically like using the option:
imapsync ... --exclude "^Files$"
If you use --office2 then imapsync will add a renaming of any
"Files" folder on host1, like using the option:
imapsync ... --f1f2 "Files=Files_renamed_by_imapsync"
The host2 account ends up with a folder named
"Files_renamed_by_imapsync", but no complaining.
R1. This folder "Files" seems to be a standard folder in Exchange Online,
but it is not. The folder contains all attachments in every email
that is in the mailbox, but without any headers.
This causes some confusion for users as these appear in their
search results as duplicate lines but without the sender details
or even the message body.
This folder seems to be usually hidden so IMAP clients can’t see it,
but for some reason sometimes it becomes visible.
R2. It looks to be a common problem with Exchange Online.
I’m not sure what causes the folder to appear.
More info here:
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
(This question/answer is taken quasi verbatim from Perttu Aaltonen)
=======================================================================
Q. Exchange fails with "BAD Command received in Invalid state".
R1. This message might happen when authenticating without ssl nor tls,
@ -386,8 +299,8 @@ R2. This message might also happen when authenticating with ssl and tls.
servers allow that.
=======================================================================
Q. From XXX to Exchange 2013 or Office365, read receipts are all
resent again after a sync. Even for old messages. How can I fix that?
Q. From XXX to Exchange 2013, read receipts are all resent again
after a sync. Even for old messages. How can I fix that?
R. imapsync can remove the header containing this read receipt request.
@ -436,7 +349,7 @@ R3. See also the FAQ FAQ.SSL_errors.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
=======================================================================
Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
Q. From XXX to Exchange 2010/2013 I get this error message
sometimes: "BAD Command Argument Error 11". What does it mean?
R. This error message comes from Exchange IMAP server when it
@ -449,12 +362,9 @@ R. This error message comes from Exchange IMAP server when it
If you can't fix this limit on Exchange then use option
--maxsize 10000000 for 10 MB, change it if needed) to tell
imapsync to skip those messages.
This value is 45 MB by default for Office365
imapsync ... --maxsize 10000000 # 10 MB for Exchange
imapsync ... --maxsize 45000000 # 45 MB for Office365
* Quota reached. The whole account is full.
It can be upped by configuration.
@ -467,10 +377,6 @@ R. This error message comes from Exchange IMAP server when it
imapsync ... --maxlinelength 9900
Office365 supports 10500 characters line length:
imapsync ... --maxlinelength 10500
In case you prefer fixing messages with long lines the hard way,
instead of skipping them with --maxlinelength 9900, just use:
@ -486,7 +392,7 @@ Exchange, which means Exchange leaves the session and say goodbye,
come back later. Rerun a sync then.
On Linux, there is a good Python script in the tarball that can
fix several things that Exchange or O365 have issues with.
fix several things that Exchange have issues with.
Use it like this:
@ -495,7 +401,7 @@ Use it like this:
It often does some miracles on messages.
=======================================================================
Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does
Q. From XXX to Exchange 2010 or 2013, the flag Flagged does
not seem to be well synced. What can I do?
R. Use the following trick. Run imapsync twice, one with --regexflag
@ -522,10 +428,9 @@ Q. How to migrate from or to Exchange 2007/2010/2013 with an
R. The tricks comes from Michele Marcionelli and Benjamin Priestman:
This doesn't work:
This may works:
imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ...
This might works:
or
imapsync ... --user2 "domain\admin2\user2" --password2 adminpassword2 ...
or
imapsync ... --user2 "admin2@domain\user2" --password2 adminpassword2 ...
@ -554,6 +459,9 @@ and b) about a mailNickname which can be different to the username.
As it seems you need to use the mailNickname (e.g. alias) defined
for user2, not the username itself.
https://techcommunity.microsoft.com/t5/exchange-team-blog/understanding-login-strings-with-pop3-imap/ba-p/610683
https://web.archive.org/web/20191014050543/https://techcommunity.microsoft.com/t5/Exchange-Team-Blog/Understanding-login-strings-with-POP3-IMAP/ba-p/610683
the second link has the same content, I give it in case the first disappears,
thanks to archive.org!
A little note from John Becker.
After you pointed out that I had issues with authorization, I realized that
@ -599,45 +507,17 @@ Hi guys. For a long time I could not start synchronization, I use admin rights
and the --authuser2 switch when I migrated from Mdaemon to Exchange2016.
I always got the error:
"Host2 failure: Error login on [my.exchange.com] with user [mydomainuser] auth [PLAIN]: 2 NO AUTHENTICATE failed."
In order for ImapSync to login to the user’s mailbox using administrator rights,
In order for ImapSync to login to the user's mailbox using administrator rights,
you need to give for the administrator the permissions to the mailbox of this
user with the command:
“Add-MailboxPermission -Identity DomainUser -User DomainAdmin -AccessRights FullAccess -InheritanceType All -AutoMapping $false”
Add-MailboxPermission -Identity DomainUser -User DomainAdmin -AccessRights FullAccess -InheritanceType All -AutoMapping $false
using Exchange Management Shell.
Without this, synchronization will not work.
Please, add this comment to the FAQ.Exchange.txt
(done!)
======================================================================
Q. How to migrate from or to Office 365 with an admin/authuser account?
Note from Yago Torres Fernandez:
(a working command using admin/authuser on host2 Office 365)
imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com ^
--password2 XXXX --ssl2 ^
but previously in Office365 you must do something like that, using powershell:
Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all
PLAIN authentication is the only way to go with --authuser1 for now.
So don't use --authmech1 SOMETHING with --authuser1 admin_user,
it will not work.
Same behavior with the --authuser2 option.
Note from Martin Paulucci:
I had to remove the domain part for the user
but not for the admin. Example:
imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated
See also:
http://www.linux-france.org/prj/imapsync_list/msg02203.html
======================================================================
Q. How to migrate from or to Exchange 2003 with an admin/authuser
account?
Q. How to migrate from or to Exchange 2003 with an admin/authuser account?
R. Thomas Edgar wrote the following
@ -711,8 +591,9 @@ R1. Be sure to have at least Exchange 2010 SP2 Rollup 5
https://tribalchicken.net/imap-sync-issues-with-exchange-2010/
R2. See also the Outlook side (thanks to Martin Hochreiter for this solution)
https://www.howto-outlook.com/faq/archivenotworking.htm
by changing Archive to sent/receive date
https://www.howto-outlook.com/faq/archivenotworking.htm
https://web.archive.org/web/20190914063435/http://www.howto-outlook.com/faq/archivenotworking.htm
It's often shorter to change one server than thousands clients
so R1 might be easier to do.

68
FAQ.d/FAQ.FirstClass.txt Normal file
View File

@ -0,0 +1,68 @@
#!/bin/cat
$Id: FAQ.FirstClass.txt,v 1.1 2021/01/26 14:03:58 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.FirstClass.txt
=======================================================================
Imapsync tips for the FirstClass IMAP server.
=======================================================================
======================================================================
Q. From FirstClass to XXX
http://www.firstclass.com/
R. Migrating from FirstClass is not easy because FirstClass, strangely,
does not show all messages via IMAP. To make it show all messages,
a trick, painful to follow by hand, is moving emails
out and back in, for each folder. May be it can be done by a script.
FirstClass releases prior to release 12 do not shows the "Sent"
folder in IMAP but FirstClass release 12 shows it.
I advice you to upgrade to FirstClass release 12 before leaving it
with imapsync or another imap tool.
Here is a command line used to migrate from FirtClass release 12:
imapsync ... \
--tmpdir /var/tmp --usecache \
--useheader Message-ID \
--idatefromheader \
--addheader \
--regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
--regextrans2 "s/[\^]/_/g" \
--regextrans2 "s/['\"\\\\]/_/g" \
--regextrans2 "s,&AC8-,-,g" \
--regextrans2 "s,&APg-,oe,g"
On Windows:
imapsync.exe ... ^
--automap ^
--usecache ^
--useheader Message-ID ^
--idatefromheader ^
--addheader ^
--regextrans2 "s,(/|^) +,$1,g" ^
--regextrans2 "s, +(/|$),$1,g" ^
--regextrans2 "s/[\^]/_/g" ^
--regextrans2 "s/['\\]/_/g" ^
--regextrans2 "s,^&AC8-,-,g" ^
--regextrans2 "s,^&APg-,oe,g"
Special thanks to Kristian Wind and Joey Alexander for helping me
writing this FAQ item.
See also this worth reading discussion in a Zimbra forum:
https://web.archive.org/web/20120627043731/http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
Another advice with FirstClass as host1, do not use --delete1emptyfolders since
it will destroy all subfolders of any empty folder. This is a FirstClass bug:
when asked to list a non-empty folder it says it has no children while it does
have ones.
See the details and explanations at
https://github.com/imapsync/imapsync/issues/207

19
FAQ.d/FAQ.Flags.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Flags.txt,v 1.21 2019/03/08 19:04:32 gilles Exp gilles $
$Id: FAQ.Flags.txt,v 1.26 2021/04/25 20:47:05 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -41,6 +41,8 @@ Q. Is it possible to sync labels and stars made by Thunderbird to
Exchange categories? Or a way in Outlook to show labels created by
Thunderbird?
Now the questions again with their answers.
=======================================================================
Q. How to debug flag issues?
@ -205,7 +207,7 @@ On Unix:
imapsync ... --regexflag 's/\\FORWARDED//gi'
or
or (on Unix)
imapsync ... --regexflag "s/\\\\FORWARDED//gi"
@ -217,11 +219,18 @@ Unix: imapsync ... --regexflag 's/\\Indexed//gi'
Windows: imapsync.exe ... --regexflag "s/\\ATTACHED//gi"
Unix: imapsync ... --regexflag 's/\\ATTACHED//gi'
All usually problematic flags in one line:
Removing usually problematic flags in one line:
Windows: imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED//gi"
Unix: imapsync ... --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED//gi'
Windows: imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi"
Unix: imapsync ... --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi'
Removing all problematic flags I've seen so far in one line:
Windows: imapsync.exe ... --regexflag "s/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g"
Unix: imapsync ... --regexflag 's/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g'
Removing all flags but the standard ones from the IMAP RFC:
Windows: imapsync.exe ... --regexflag "s/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg"
Unix: imapsync ... --regexflag 's/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg'
=======================================================================
Q. How to convert flags with $ to \ character?

283
FAQ.d/FAQ.Folders_Mapping.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Folders_Mapping.txt,v 1.20 2019/07/27 20:04:57 gilles Exp gilles $
$Id: FAQ.Folders_Mapping.txt,v 1.23 2021/04/28 23:42:04 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -11,25 +11,93 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
Imapsync tips about changing folders names.
=======================================================================
By default, imapsync syncs all folders, one by one, in alphanumeric order.
Questions answered in this FAQ are:
By default, folders names from host1 are reproduced identical on host2,
except for the prefix and the separator which are automatically adapted
for host2.
Q. What is the imapsync default behaviour with folders?
The IMAP protocol has a specific way to code folders names,
Q. What encoding imapsync uses to deal with folder names?
Q. How can I encode folder names for imapsync without getting a headache?
Q. What are the easy ways to change folder names?
Q. What should I know before playing with the --regextrans2 option?
Q. How to remove the string "INBOX." in the name of destination folders?
Q. How to change only INBOX name to Inbox_Migrated?
Q. How to sync all folders to a single INBOX folder?
Q. How to sync all folders to a single WHOLE folder?
Q. How to sync a complete account in a subfolder called FOO?
Q. How to substitute all characters dot "." by underscores "_" ?
Q. How to substitute all doublequotes " by underscores _ ?
Q. How to substitute all characters * % . and # by underscores _ ?
Q. How to change folder names like this:
[mail/Sent Items] -> [Sent]
[mail/Test] -> [INBOX/Test]
[mail/Test2] -> [INBOX/Test2]
Q. Is it possible to synchronize all messages from one server to
another without recreating the folder structure and the target server.
Q. I would like to move emails from InBox to a sub-folder called,
say "2010-INBOX" based on the date. Like all emails received in the
year 2010 should be moved to the folder called "2010-INBOX".
Q. How to sync a complete account in a subfolder called FOO,
the hard way?
Q. How to transform all folder names to lowercase?
Q. How to transform all folder names to uppercase?
Q. How to transform all folder names to lowercase, except the first
character in uppercase?
Now the questions again with their answers.
=======================================================================
Q. What is the imapsync default behaviour with folders?
R. Imapsync syncs all folders, one by one, in alphanumeric order,
reproducing their name identical from the source account at host1 to
the destination account at host2, except for the prefix and the
separator which are automatically adapted for host2.
=======================================================================
Q. What encoding imapsync uses to deal with folder names?
R. The IMAP protocol has a specific way to code folders names,
especially when these names use non-ascii 7bit characters.
This encoding is called utf7imap.
Imapsync uses the same encoding as IMAP: utf7imap.
Imapsync uses the same encoding as IMAP, it uses utf7imap.
In order to well specify folders names on the command line, imapsync
prints the complete folder list of both sides at the beginning of each run.
=======================================================================
Q. How can I encode folder names for imapsync without getting a headache?
R. In order to well specify folders names on the command line, imapsync
prints the complete folder list of both sides at the beginning
of each run.
The left column is the encoding you have to use, without the first enclosing
square brackets [], the right column is the human utf8 view.
This document gives most examples with the powerful but complex
=======================================================================
Q. What are the easy ways to change folder names?
R. This document gives most examples with the powerful but complex
option --regextrans2. Before using --regextrans2 you should
consider using --automap and --f1f2 because they are simpler
to understand and use.
@ -47,8 +115,10 @@ to understand and use.
Add --debug to see what's really going on.)
=======================================================================
Q. What should I know before playing with the --regextrans2 option?
Things to know and understand before playing with --regextrans2
The things to know and understand before playing with --regextrans2 are:
*) --regextrans2 is used to transform folders names.
@ -101,39 +171,60 @@ Here
ETA calculation and it's a supplementary check on folders.
=======================================================================
Q. Give examples about --regextrans2
Q. How to remove the string "INBOX." in the name of destination folders?
Examples:
R. To remove "INBOX." in the name of destination folders:
1) To remove INBOX. in the name of destination folders
On Linux:
imapsync ... --regextrans2 's/^INBOX\.(.+)/$1/'
2) To change only INBOX to Inbox_Migrated
On Windows:
imapsync ... --regextrans2 "s/^INBOX\.(.+)/$1/"
=======================================================================
Q. How to change only INBOX name to Inbox_Migrated?
R.
imapsync ... --regextrans2 's{^INBOX$}{Inbox_Migrated}'
2a) To sync all folders to INBOX
=======================================================================
Q. How to sync all folders to a single INBOX folder?
R.
imapsync ... --regextrans2 "s/.*/INBOX/"
=======================================================================
Q. How to sync all folders to a single WHOLE folder?
2b) To sync a complete account in a subfolder called FOO
R.
With imapsync release 1.641 and former, simply use
imapsync ... --regextrans2 "s,.*,WHOLE,"
=======================================================================
Q. How to sync a complete account in a subfolder called FOO?
R. With imapsync release 1.641 and former, simply use:
imapsync ... --subfolder2 FOO
Examples to this subfolder problem for any release
Examples to this subfolder problem for previous releases
can be seen below, in another Q/R section.
3) to substitute all characters dot "." by underscores "_"
=======================================================================
Q. How to substitute all characters dot "." by underscores "_" ?
R.
--regextrans2 "s,\.,_,g"
3b) to substitute all doublequotes " by underscores _
=======================================================================
Q. How to substitute all doublequotes " by underscores _ ?
On Linux/Unix:
@ -143,7 +234,8 @@ On Windows:
--regextrans2 s,\^",_,g
3c) to substitute all characters * % . and # by underscores _
=======================================================================
Q. How to substitute all characters * % . and # by underscores _ ?
On Linux/Unix:
@ -154,13 +246,14 @@ On Windows:
--regextrans2 "tr,*%.#,_,"
You can increase the *%.# list by any unwanted character,
plus, the order doesn't count. See 3d) for &
plus, the order doesn't count. See below for &
3d) It is a bad idea to substitute & characters since &
It is a bad idea to substitute & characters since &
is a character to encode non-ascii characters in IMAP folder names.
4) to change folder names like this:
=======================================================================
Q. How to change folder names like this:
[mail/Sent Items] -> [Sent]
[mail/Test] -> [INBOX/Test]
[mail/Test2] -> [INBOX/Test2]
@ -170,12 +263,56 @@ On Linux/Unix:
--regextrans2 's,^mail/Sent Items$,Sent,' \
--regextrans2 's,^mail/,INBOX/,'
a) I want folders 'Sent', 'Sent Messages', 'Gesendet' from host1
to be merged to folder 'Sent' on host2.
--regextrans2 "s{Sent|Sent Messages|Gesendet}{Sent}"
b) I do not want this rule a) to apply to any subfolders,
e.g. 'folder/Sent' or 'folder/Sent Messages' etc. but only
basefolders.
So you have to add an anchor character ^ meaning "beginning of the string",
and an anchor character $ meaning "end of the string" like this:
--regextrans2 's{^Sent$|^Sent Messages$|^Gesendet$}{Sent}'
c) I want just a) to be combined with a subfolder, for example:
INBOX -> Subfolder/INBOX
Sent -> Subfolder/Sent
Sent Messages -> Subfolder/Sent
Gesendet -> Subfolder/Sent
folder/Sent -> Subfolder/folder/Sent
folder/Sent Messages -> Subfolder/folder/Sent
folder/Gesendet -> Subfolder/folder/Sent
To achieve this, use:
--subfolder2 Subfolder --regextrans2 's{Sent|Sent Messages|Gesendet}{Sent}'
d) I want a) and b) to be combined with a subfolder, for example:
INBOX -> Subfolder/INBOX
Sent -> Subfolder/Sent
Sent Messages -> Subfolder/Sent
Gesendet -> Subfolder/Sent
folder/Sent -> Subfolder/folder/Sent
folder/Sent Messages -> Subfolder/folder/Send Messages
folder/Gesendet -> Subfolder/folder/Gesendet
...
This one is trickier to achieve:
--subfolder2 Subfolder --regextrans2 's{^Subfolder/Sent$|^Subfolder/Sent Messages$|^Subfolder/Gesendet$}{Subfolder/Sent}'
======================================================================
Q. Is it possible to synchronize all messages from one server to
another without recreating the folder structure and the target server.
another without recreating the folder structure and the target server.
R. Yes.
R. Yes. This question has already been answered but a little of
redundancy can't hurt, can he?
For example, to synchronize all messages in all folders on host1
to folder INBOX only on host2:
@ -225,9 +362,10 @@ b) Manually:
3) Cut/paste in INBOX.2010-INBOX
=======================================================================
Q. How to sync a complete account in a subfolder called FOO,
the hard way.
the hard way?
R0. See the easy above using --subfolder2
@ -259,22 +397,93 @@ On Windows:
--regextrans2 "s,(.*),FOO/$1,"
or:
=======================================================================
Q. How to transform all folder names to lowercase?
c) Any separator, any prefix solution, FOO is the subfolder:
R.
It is a complicated line because every case is taken into account.
Type it in one line (or with the \ at the end of first line on Unix shells.
First, some documentation to understand how the regexes work.
perldoc perlre says:
On Linux/Unix:
\l lowercase next char (think vi)
\u uppercase next char (think vi)
\L lowercase till \E (think vi)
\U uppercase till \E (think vi)
\E end case modification (think vi)
\Q quote (disable) pattern metacharacters till \E
--regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \
--regextrans2 's,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,'
On Linux:
If the separator on host2 is the character slash "/", use:
--regextrans2 's,([^/]+),\L$1\E,g'
If the separator on host2 is the character dot ".", use:
--regextrans2 's,([^.]+),\L$1\E,g'
On Windows:
--regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," ^
--regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,"
If the separator on host2 is the character slash "/", use:
--regextrans2 "s,([^/]+),\L$1\E,g"
If the separator on host2 is the character dot ".", use:
--regextrans2 "s,([^.]+),\L$1\E,g"
=======================================================================
Q. How to transform all folder names to uppercase?
R.
On Linux:
If the separator on host2 is the character slash "/", use:
--regextrans2 's,([^/]+),\U$1\E,g'
If the separator on host2 is the character dot ".", use:
--regextrans2 's,([^.]+),\U$1\E,g'
On Windows:
If the separator on host2 is the character slash "/", use:
--regextrans2 "s,([^/]+),\U$1\E,g"
If the separator on host2 is the character dot ".", use:
--regextrans2 "s,([^.]+),\U$1\E,g"
=======================================================================
Q. How to transform all folder names to lowercase, except the first
character in uppercase?
R.
On Linux:
If the separator on host2 is the character slash "/", use:
--regextrans2 's,([^/]+),\u\L$1\E,g'
If the separator on host2 is the character dot ".", use:
--regextrans2 's,([^.]+),\u\L$1\E,g'
On Windows:
If the separator on host2 is the character slash "/", use:
--regextrans2 "s,([^/]+),\u\L$1\E,g"
If the separator on host2 is the character dot ".", use:
--regextrans2 "s,([^.]+),\u\L$1\E,g"
=======================================================================

102
FAQ.d/FAQ.Folders_Selection.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Folders_Selection.txt,v 1.8 2019/07/27 20:04:46 gilles Exp gilles $
$Id: FAQ.Folders_Selection.txt,v 1.11 2020/12/31 11:41:46 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -10,6 +10,34 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
Imapsync tips to select folders.
=======================================================================
Questions answered in this FAQ are:
Q. What is the default imapsync behaviour about folders?
Q. What are the options to select folders?
Q. Is it ok to use --subfolder1 and --subfolder2 to just select folders?
Q. How can I sync only one folder?
Q. What are --subscribe and --subscribed for, and how can they be used?
Q. How to exclude a folder hierarchy like "public"?
Q. How to exclude only the INBOX folder?
Q. How to exclude folders matching SPAM no matter the case,
aka, how to be case insensitive?
Q. How to make option --folder "MyFolder" be recursive?
Q. How to change folders names or move them elsewhere?
Now the questions again with their answers.
=======================================================================
Q. What is the default imapsync behaviour about folders?
By default, Imapsync syncs all folders, one by one, in alphanumeric order.
The IMAP protocol has a specific way to code folders names,
@ -25,6 +53,52 @@ The left column is the encoding you have to use, without the first enclosing
square brackets [], the right column is the human utf8 view.
=======================================================================
Q. What are the options to select folders?
R. From https://imapsync.lamiral.info/README
OPTIONS/folders
--folder str : Sync this folder.
--folder str : and this one, etc.
--folderrec str : Sync this folder recursively.
--folderrec str : and this one, etc.
--folderfirst str : Sync this folder first. Ex. --folderfirst "INBOX"
--folderfirst str : then this one, etc.
--folderlast str : Sync this folder last. --folderlast "[Gmail]/All Mail"
--folderlast str : then this one, etc.
--nomixfolders : Do not merge folders when host1 is case-sensitive
while host2 is not (like Exchange). Only the first
similar folder is synced (example: with folders
"Sent", "SENT" and "sent"
on host1 only "Sent" will be synced to host2).
--skipemptyfolders : Empty host1 folders are not created on host2.
--include reg : Sync folders matching this regular expression
--include reg : or this one, etc.
If both --include --exclude options are used, then
include is done before.
--exclude reg : Skips folders matching this regular expression
Several folders to avoid:
--exclude 'fold1|fold2|f3' skips fold1, fold2 and f3.
--exclude reg : or this one, etc.
=======================================================================
Q. Is it ok to use --subfolder1 and --subfolder2 to just select folders?
R. No.
If you do:
imapsync ... --subfolder1 Foo --subfolder2 Foo
then it ends up with all messages from Foo going to Foo/INBOX
=======================================================================
Q. How can I sync only one folder?
@ -56,23 +130,15 @@ R. In the IMAP protocol each user can subscribe to one or more folders.
=======================================================================
Q. I want to exclude a folder hierarchy like "public"
Q. How to exclude a folder hierarchy like "public"?
R. Use:
--exclude "^public\."
or maybe
--exclude '^"public\.'
In the example given the character "." is the folder separator, you
can omit it. Just take the string as it appears on the imapsync
output line :
From folders list : [INBOX] [public.dreams] [etc.]
--exclude "public"
=======================================================================
Q. I want to exclude only INBOX
Q. How to exclude only the INBOX folder?
R. Use:
@ -83,8 +149,8 @@ A good way to see what will be done is to first use:
imapsync ... --exclude "^INBOX$" --justfolders --nofoldersizes --dry
=======================================================================
Q. I want to exclude folders matching SPAM no matter the case,
aka how to be case insensitive
Q. How to exclude folders matching SPAM no matter the case,
aka, how to be case insensitive?
R. Use:
@ -96,7 +162,7 @@ A good way to see what will be done is to first use:
=======================================================================
Q. I want the --folder "MyFolder" option be recursive.
Q. How to make option --folder "MyFolder" be recursive?
Two solutions:
@ -108,5 +174,11 @@ R2. Use --include "^MyFolder"
Then the folder "MyFolder" and all its subfolders will be handled
and only them.
=======================================================================
Q. How to change folders names or move them elsewhere?
R. Read the document FAQ.Folders_Mapping.txt also available at
https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
=======================================================================
=======================================================================

30
FAQ.d/FAQ.Folders_Sizes.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Folders_Sizes.txt,v 1.3 2019/07/25 15:57:04 gilles Exp gilles $
$Id: FAQ.Folders_Sizes.txt,v 1.4 2021/01/26 13:05:21 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -13,13 +13,13 @@ Questions answered in this FAQ are:
Q. How can I know the size of all folders before launching a sync?
Q. The folders sizes and the number of messages don't match between
the source mailbox and the destination mailbox.
Why and what can I do?
Q. After a sync, the folders sizes and the number of messages don't
match between the source mailbox on host1 and the destination
mailbox on host2.
Why? What can I do about it?
Now the questions again with their answers.
=======================================================================
Q. How can I know the size of all folders before launching a sync?
@ -27,14 +27,18 @@ R. Use --justfoldersizes option.
With the --justfoldersizes option, imapsync evaluates and displays the
folders sizes on both sides and then exits without doing any transfer.
It also presents the total size of each account, the number of messages,
the biggest message size, the number of folders.
Example:
imapsync ... --justfoldersizes
=======================================================================
Q. The folders sizes and the number of messages don't match between
the source mailbox on host1 and the destination mailbox on host2.
Why and what can I do?
Q. After a sync, the folders sizes and the number of messages don't
match between the source mailbox on host1 and the destination
mailbox on host2.
Why? What can I do about it?
R. There are several reasons to explain the different counts between
source and destination after a sync.
@ -66,11 +70,13 @@ R. There are several reasons to explain the different counts between
6) Cosmic rays...
The important part to take into account is the last lines of imapsync
if you encounter the line:
"The sync looks good, all X identified messages in host1 are on host2."
and also no unidentified messages.
The important part to take into account for a good and well done sync
is the last lines of imapsync. A good sync is when you encounter the
three final lines:
"The sync looks good, all xxxx identified messages in host1 are on host2."
"There is no unidentified message on host1."
"Detected 0 errors"
=======================================================================
=======================================================================

98
FAQ.d/FAQ.General.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: FAQ.General.txt,v 1.242 2019/12/11 20:28:33 gilles Exp gilles $
# $Id: FAQ.General.txt,v 1.244 2021/02/19 13:38:39 gilles Exp gilles $
=======================================================================
General FAQ for imapsync
@ -21,6 +21,8 @@ Q. What are the most important differences between the Unix shell syntax
Q. How to install imapsync?
Q. How to upgrade imapsync?
Q. How to use imapsync?
Q. Can you give some configuration examples?
@ -77,17 +79,10 @@ Q. Is it possible to sync also the UIDs of the IMAP server?
Q. The option --subscribe does not seem to work
Q. With huge account (many messages) when it comes to reading the
destination server it comes out this error:
"To Folder [INBOX.foobar] Not connected"
What can I do?
Q. Can Imapsync filter Spam during the sync?
Q. How to migrate from uw-imap with an admin/authuser account?
Q. How to migrate from cyrus with an admin account?
Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
Netscape servers with an admin account?
@ -103,9 +98,6 @@ Q. I want to play with headers line and --regexmess but I want to leave
Q. My imap server does not accept a message and warns
"Invalid header". What is the problem?
Q. The contact folder isn't well copied.
How to copy the contact folder?
Q: How can I write an .rpm with imapsync
Q. Where I can read up on the various IMAP RFCs?
@ -179,10 +171,16 @@ any \$1 by $1 (remove the \ before $).
=======================================================================
Q. How to install imapsync?
R. Read the INSTALL files in the tarball. also available at
https://imapsync.lamiral.info/#doc
R. Read the INSTALL files in the tarball. They are also available at
https://imapsync.lamiral.info/#install
and
https://imapsync.lamiral.info/INSTALL.d/
=======================================================================
Q. How to upgrade imapsync?
R. Do like a fresh installation.
=======================================================================
Q. How to use imapsync?
@ -416,6 +414,7 @@ R3 If you want to install the Perl module locally in a directory
Q. How can I use imapsync with Mail::IMAPClient 2.2.9 perl module?
R. Mail::IMAPClient 2.2.9 is no longer supported.
Help yourself.
=======================================================================
Q. How to verify imapsync.exe I got is the right file bit per bit?
@ -512,8 +511,8 @@ R. imapsync doesn't talk POP3 but I think you mean UID in IMAP.
Q. Is it possible to sync also the UIDs of the IMAP server?
R. UIDs in IMAP are chosen and created by the server, not by the
client software. imapsync is a client software. So UIDs cannot
be synced by any imap method.
client software. imapsync is a client software command.
So UIDs cannot be synced by any imap method.
UIDs may be synced via a rsync command between the imap servers but
it implies they are the same software, among other constraints.
@ -526,16 +525,6 @@ R1. Use it with --subscribed
R2. There is also the --subscribe_all option that subscribe
to all folders on host2.
=======================================================================
Q. With huge account (many messages) when it comes to reading the
destination server it comes out this error:
"To Folder [INBOX.foobar] Not connected"
What can I do?
R. May be spending too much time on the source server, the connection
timed out on the destination server.
Try options --nofoldersizes
======================================================================
Q. Can Imapsync filter Spam during the sync?
@ -560,46 +549,6 @@ R. Use the following syntax:
imapsync ... --user1="loginuser*admin_user" --password1 "admin_user_password"
======================================================================
Q. How to migrate from cyrus with an admin account?
R. Use:
imapsync ... \
--authuser1 admin_user ----password1 admin_user_password \
--user1 foo_user --ssl1
Instead of --ssl1 the alternative --tls1 can be used.
With --authuser1, the option --authmech1 PLAIN is set
automatically, you don't have to add it.
PLAIN authentication is the only way to go with --authuser1 for now.
So don't use --authmech1 SOMETHING with --authuser1 admin_user,
it will not work.
Same behavior with the --authuser2 option.
Do not forget the option --ssl1 or --tls1 since PLAIN auth is only
supported with ssl encryption most of the time. But it can
work without --ssl1 nor --tls1 if PLAIN is permitted in clear text
transmissions (the normal mode).
Add the AdminAccount to admins line in /etc/imapd.conf
Give AdminAccount lrswipkxtecda to the Cyrus Imap account
being migrated from, "joe" here.
Here is an example:
imapsync \
--host1 server1 \
--user1 joe \
--authuser1 AdminAccount \
--password1 AdminAccountPassword \
--ssl1 \
--host2 server2 \
--user2 joe \
--password2 joespassonserver2 \
--exclude "^user\."
======================================================================
Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
Netscape servers with an admin account?
@ -717,22 +666,9 @@ Solutions c) and d) keep "From " lines information
(normally it's useless to keep them)
Best solutions are e) or d).
=======================================================================
Q. The contact folder isn't well copied.
How to copy the contact folder?
R. Forget the destination server (choose the same)
Change the script around line 1426
# ITSD
$new_id = $from->copy($t_fold,$f_msg);
#$new_id = $to->append_string($t_fold,$string, $flags_f, $d);
and tried a copy of the mail instead an append_string. Because we are
using the same server, we can use $from->copy Therefore we seem to not
download and upload the message and therefore we do not have any
format issues. And now it works fine. (Thanks to Hansjoerg.Maurer)
I prefer the d) solution because it fixes the issue
and keeps the old "From" value while the e) solution removes
the old "From" value definitively.
======================================================================
Q: How can I write an .rpm with imapsync

316
FAQ.d/FAQ.Gmail.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Gmail.txt,v 1.65 2019/12/04 18:26:00 gilles Exp gilles $
$Id: FAQ.Gmail.txt,v 1.79 2021/03/31 16:20:28 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -11,6 +11,7 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Gmail accounts?
Q. Even after several syncs the number of messages differs
@ -19,6 +20,10 @@ Q. Even after several syncs the number of messages differs
Q. How many days does it take to transfer X GB?
Q. What is the Gmail free maximum storage size?
R. 15 GB in 2020. Need more?
https://gsuite.google.com/pricing.html
Q. How to synchronize from Gmail to Gmail?
Q. How to synchronize from XXX to Gmail?
@ -26,12 +31,25 @@ Q. How to synchronize from XXX to Gmail?
Q. How to synchronize from Gmail to XXX?
Q. I can't authenticate with Gmail via IMAP
and Gmail says "Please log in via your web browser"
and Gmail says
"Please log in via your web browser"
or it says
"Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)"
Q. Can not open imap connection on [imap.gmail.com]
Unable to connect to imap.gmail.com
Q. Can I transfer "Chat" messages with imapsync?
Q. I want to sync the Sent folder only, what is it on Gmail?
Q. The context is a Gmail account at the source, but not Gmail at the
destination. I want cross-duplicates, different Gmail folders/labels
generating duplicate messages across folders on the destination,
except for the "All Mail" folder.
Q. What are the Gmail imap names of the folders
"Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"?
Q. Can I transfer Gmail "Chat" messages with imapsync?
Q. Can I safely use --useuid for Gmail transfers?
@ -62,30 +80,70 @@ Q. Can I use imapsync to transfer from or to Gmail accounts?
R. Yes. But IMAP access to a Gmail account is not allowed by default so
it has to be allowed in the Gmail configuration part:
-> Settings
-> Forwarding and POP/IMAP
-> IMAP Access
-> Enable IMAP
-> Do not go to "Manage your Google Account" (the upper right icon with your Google face)
-> Instead, go to Gmail at https://mail.google.com/
-> Click on Settings, the gear button on the upper right of Gmail.
-> Click on "See all settings" at the top of the pull down menu.
-> Go to the tab "Forwarding and POP/IMAP"
-> Go down to "IMAP Access"
-> Activate the radio button "Enable IMAP"
-> Click on the button "Save changes" at the bottom of the page.
Also, there is the authentication issue to consider.
That's not finished, there is also the authentication issue to consider.
There are three different ways to authenticate to Gmail with Imapsync,
either:
* "Access for less secure apps"
* "2-step verification"
* XOauth
1) "Access for less secure apps"
2) "2-step verification"
3) XOauth2 (not ready for individual Gmail authentication, only for groups)
For the last one, XOauth, see the document
For the last one, XOauth2, see the document
https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
1) "Access for less secure apps" is turned ON
1) "Access for less secure apps", the first way to authenticate
to Gmail with Imapsync. This is the easiest way because
you just have to turn one button on.
Go to https://myaccount.google.com/
Click on the Security tab on the left menu or go directly to
https://myaccount.google.com/security?gar=1
-> Verify 2-step verification is OFF
https://myaccount.google.com/security?hl=en&pli=1&nlr=1#signin
Near the end of the page,
-> Turn ON "Access for less secure apps"
https://www.google.com/settings/security/lesssecureapps
https://support.google.com/accounts/answer/6010255?hl=en
Here is what Google add on this "Access for less secure apps":
"Some apps and devices use less secure sign-in technology, which makes your
account vulnerable. You can turn off access for these apps, which we recommend,
or turn it on if you want to use them despite the risks.
Google will automatically turn this setting OFF if it’s not being used."
So after a while, this button will be turned off again.
I suggest to turn it off once the job with imapsync is finished.
If you forgot to also enable imap you end up with login failure like this one:
"Host failure: Error login on [imap.gmail.com] with user [xxxx@gmail.com] auth [LOGIN]:
NO [ALERT] Your account is not enabled for IMAP use. Please visit your Gmail
settings page and enable your account for IMAP access. (Failure)"
After using imapsync on a Gmail account, sometimes you may encounter an email
with the title "Critical security alert for your linked Google Account"
and the body
"Sign-in attempt was blocked for your linked Google Account
xxxx@gmail.com
Someone just used your password to try to sign in to your account from
a non-Google app. Google blocked them, but you should check what happened.
Review your account activity to make sure that no one else has access."
Click on the button "Check Activity" and let Gmail know you were the author
of the sign in.
Sometimes, with Gsuite, the login for the users are ok with imapsync
for a while then they become forbidden with
"failure: Error login on [imap.gmail.com] with user [xxx] auth [LOGIN]:
@ -95,10 +153,28 @@ In that case, use:
https://admin.google.com/AdminHome?hl=fr#ServiceSettings/notab=1&service=securitysetting&subtab=lesssecureappsaccess
to allow "Access for less secure apps to all users"
(Thanks to Sébastien R. for this input)
Remark: even if the user has enabled less secure apps at Super Admin
and reseller level, it still needs amended at
https://myaccount.google.com/security
(Thanks Tony@gle for this last tip!)
2) "2-step verification" is turned ON
Follow
2) "2-step verification" the second way to authenticate
to Gmail with Imapsync.
Go to
https://myaccount.google.com/security
and sign in with the account you want to transfer email to or from.
Go to the "Signing in to Google" part of the security page and follow
the process to activate the "2-step verification" until it is ok.
Then, just below "2-step verification" click on the "App passwords"
and follow the instruction to get a 16 characters password, on the
pull down button "Select app", choose "Other (Custom name)" and call
it imapsync for example. Now use this this password to access the
mailbox with imapsync.
The following document details what I've just described:
https://support.google.com/accounts/answer/185833?hl=en
Even if Google says "Every App Password is only used once",
it's false and the password can be used many times.
@ -122,18 +198,25 @@ Q. How many days does it take to transfer X GB?
R. Basically it takes X days to transfer X GB per account.
Gmail has usage limits per day and use throttles when
they are overtaken
http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
https://support.google.com/a/answer/1071518?hl=en
From the previous link:
* it's 2X days to upload X GB to Gmail, it's why I suggest to add
--maxbytespersecond 10000 --maxbytesafter 1_000_000_000
for uploading messages to Gmail
* it's X/2 days to download X BG from Gmail, it's why I suggest to add
* Upload with IMAP: 500 MB per day.
It's 2X days to upload X GB to Gmail, it's why I suggest to add
--maxbytespersecond 20000 --maxbytesafter 1_000_000_000
for downloading messages from Gmail
for uploading messages to Gmail
(should be --maxbytespersecond 5787 --maxbytesafter 500_000_000
but the higher values given work well)
That's theoretical values that always work in practice. Try
upper values and see if they still work.
* Download with IMAP: 2500 MB per day.
It's X/2.5 days to download X GB from Gmail, it's why I suggest to add
--maxbytespersecond 40_000 --maxbytesafter 3_000_000_000
for downloading messages from Gmail
(should be --maxbytespersecond 28_935 --maxbytesafter 2_500_000_000
That's empirical values that work in practice.
Try upper values and see if they still work.
I would be grateful if you report them to me.
How Gmail says limits are reached?
This is either a disconnection with
@ -287,12 +370,11 @@ be 500_000_000, not 1_000_000_000, but 1GB is usually ok.
--maxsleep 2 is to avoid sleeping more than 2 seconds when
the --maxbytespersecond value implies a long sleep between 2 copies.
--maxsize 25_000_000 is mandatory since Gmail limits messages size
up to 25 MB. This value increases over time, it was 10 MB some
years ago so you can try higher values. The Gmail page about
this limit is https://support.google.com/mail/answer/6584
--maxsleep 0 cancels any imapsync bandwidth restriction given by
the options --maxbytespersecond or --maxmessagespersecond
It does not cancel the imap server bandwidth restrictions.
--automap is optional but it will save manual folder names
--automap is optional but it will save manual folders names
changes or the use of --regextrans2 or --f1f2 to map folder names.
--expunge1 is optional. It deletes messages marked \Deleted on host1.
@ -347,6 +429,7 @@ imapsync \
--password1 gmailsecret \
--host2 localhost \
--user2 tata \
--password2 tatasecret \
--gmail1
Or, replacing what does --gmail1 (values are from imapsync 1.970):
@ -360,7 +443,7 @@ imapsync \
--user2 tata \
--password2 tatasecret \
--maxbytespersecond 40_000 \
--maxbytesafter 2_500_000_000
--maxbytesafter 2_500_000_000 \
--automap \
--maxsleep 2 \
--skipcrossduplicates \
@ -384,48 +467,51 @@ allow you to use an upper value than 2 GBytes per 24h without
disconnections.
--useheader="X-Gmail-Received" --useheader "Message-Id" are not
mandatory. I use them because I found (several years ago, it may
have changed) that Gmail always adds a different header
"X-Gmail-Received:" to all messages it gets. So the identification
by imapsync can not fail using this header. "Message-Id" is there
for safety about this Gmail rule.
mandatory. I use them because I found (several years ago, it may have
changed) that Gmail always adds a different header "X-Gmail-Received:"
to all messages it gets. So the identification by imapsync can not
fail using this header. "Message-Id" is there for safety about this
Gmail rule.
--automap is optional but it will save manual folder names
changes or the use of --regextrans2 to map folder names.
--automap is optional but it will save manual folder names changes or
the use of --regextrans2 to map folder names.
--regextrans2 "s,\[Gmail\].,,"
If your destination imap server doesn't like "[Gmail]" name,
If your destination imap server doesn't like the "[Gmail]" name,
get rid of this "[Gmail]" part with that.
You can select folders exported to imap within the gmail preferences,
for example you may unselect all "System labels".
You can select folders exported to imap within the Gmail preferences,
for example, you may unselect all "System labels".
--skipcrossduplicates is optional but it can save Gigabytes of hard
disk memory. Within imap protocol, Gmail presents Gmail labels as
folders, so a message labeled "Work" "ProjectX" "Urgent" ends up
in three different imap folders "Work" "ProjectX" and "Urgent"
after an imap sync. Option --skipcrossduplicates prevent this behavior.
folders, so a message labelled "Work" "ProjectX" "Urgent" ends up in
three different imap folders "Work" "ProjectX" and "Urgent" after an
imap sync. Option --skipcrossduplicates prevent this behavior.
An issue with --skipcrossduplicates is that the first label synced
by imapsync goes to its corresponding folder but other labels are
then ignored. You can choose what labels have the
priority by using the --folderfirst option. For example
--folderfirst "Work" will sync messages labeled "Work" before
messages labeled "CanWait" or "Urgent". By default imapsync
syncs folders (Gmail labels) using the classical alphanumeric order.
An issue with --skipcrossduplicates is that the first label synced by
imapsync goes to its corresponding folder but other labels are then
ignored. You can choose what labels have priority by using the
--folderfirst option. For example, --folderfirst "Work" will sync
messages labelled "Work" before messages labelled "CanWait" or
"Urgent". By default imapsync syncs folders (Gmail labels) using the
classical alphanumeric order.
--folderlast "CanWait" will sync only messages that have the
label CanWait and only it.
Option --folderlast "CanWait" will sync only messages that only have
the label CanWait.
--folderlast "[Gmail]/All Mail", in conjunction with option
--skipcrossduplicates, will only put in "[Gmail]/All Mail"
the messages that are not labeled at all.
Option --folderlast "[Gmail]/All Mail", in conjunction with option
--skipcrossduplicates will only put in "[Gmail]/All Mail" the messages
that are not labelled at all.
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
=======================================================================
Q. I can't authenticate with Gmail via IMAP
and Gmail says "Please log in via your web browser"
and Gmail says
"Please log in via your web browser"
or it says
"Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)"
R0. My current settings on two Gmail accounts are made
with three steps 0) 1) 2)
@ -441,6 +527,7 @@ https://support.google.com/accounts/answer/6010255?hl=en
R1. See Coert Grobbelaar solution:
https://web.archive.org/web/20150906230041/http://security.stackexchange.com/questions/86404/how-do-i-interact-with-google-to-import-email-via-imapsync
https://support.google.com/accounts/answer/185833
R2. I had the same issue one time (mars 2015) logging to Gmail
with imapsync.
@ -450,6 +537,48 @@ it asked me to receive a code via a mobile, I said yes,
I entered the code and everything went ok on next syncs
with imapsync.
R1 again. Since people rarely follow links in a txt file, I reproduce
Google documentation from the link
https://support.google.com/accounts/answer/185833
Sign in using App Passwords
An App Password is a 16-digit passcode that gives a non-Google app or
device permission to access your Google Account.
App Passwords can only be used with accounts that
have 2-Step Verification turned on.
When to use App Passwords
To help keep your account secure, use "Sign in with Google" to
connect apps to your Google Account.
If the app you’re using doesn’t offer this option, you can either:
Use App Passwords to connect to your Google Account
Switch to a more secure app
Create & use App Passwords
Note: If you use 2-Step-Verification
( https://support.google.com/accounts/answer/185839 )
and are seeing a "password incorrect" error when trying to access
your Google Account, an App Password may solve the problem.
* Go to your Google Account https://myaccount.google.com/
* On the left navigation panel, choose Security.
* On the "Signing in to Google" panel, choose App Passwords.
If you don’t see this option:
* 2-Step Verification is not set up for your account.
* 2-Step Verification is set up for security keys only.
* Your account is through work, school, or other organization.
* You’ve turned on Advanced Protection for your account.
* At the bottom, choose Select app and choose the app you’re using.
* Choose Select device and choose the device you’re using.
* Choose Generate.
* Follow the instructions to enter the App Password.
The App Password is the 16-character code in the yellow bar on your device.
* Choose Done.
Most of the time, you’ll only have to enter an App Password once per
app or device, so don’t worry about memorizing it.
=======================================================================
Q. Can not open imap connection on [imap.gmail.com]:
@ -483,8 +612,83 @@ Or go to http://ping.eu/nslookup/ to get the resolution.
Thanks to Chris Nolan to report, understand and fix this issue!
=======================================================================
Q. Can I transfer "Chat" messages with imapsync?
Q. I want to sync the Sent folder only, what is it on Gmail?
R1. Use:
imapsync ... --folder "[Gmail]/Sent Mail"
R2. In fact, by configuration from the user, it can be another folder
name presented in IMAP. Use --automap --dry to see this mapping.
See below
=======================================================================
Q. The context is a Gmail account at the source, but not Gmail at the
destination. I want cross-duplicates, different Gmail folders/labels
generating duplicate messages across folders on the destination,
except for the "All Mail" folder.
R. This behavior, different Gmail labels generating duplicates across
folders except for the "All Mail" folder, is obtained with a
synchronization with no --gmail1 but instead --exclude "All Mail"
followed by another synchronization with --gmail1 but without
--exclude "All Mail".
imapsync ... --exclude "All Mail"
imapsync ... --gmail1
This second sync is done to get the messages with no labels at all
from the folder "All Mail", if they are some of them.
Technically speaking, the first synchronization has not the option
--skipcrossduplicates but the second one has the option
--skipcrossduplicates and it also performs the Gmail "All Mail"
folder/label at the end so only the messages that have no labels will
end up in the "All Mail" destination folder (which is not Gmail,
remember the context).
Thanks to Mike Alexander for this suggestion.
Mike also provided a patch to implement that directly in imapsync.
See https://github.com/imapsync/imapsync/pull/266
(I refused it because well maintaining it will be very hard for me
and I have other priorities).
=======================================================================
Q. What are the Gmail imap names of the folders
"Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"?
R. By default, Gmail imap names of the folders
"Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"
are like this:
Turned on automapping folders ( use --noautomap to turn off automapping )
Host1: special [Gmail]/All Mail = \All
Host1: special [Gmail]/Drafts = \Drafts
Host1: special [Gmail]/Sent Mail = \Sent
Host1: special [Gmail]/Spam = \Junk
Host1: special [Gmail]/Starred = \Flagged
Host1: special [Gmail]/Trash = \Trash
Host2: special [Gmail]/All Mail = \All
Host2: special [Gmail]/Bin = \Trash
Host2: special [Gmail]/Drafts = \Drafts
Host2: special [Gmail]/Sent Mail = \Sent
Host2: special [Gmail]/Spam = \Junk
Host2: special [Gmail]/Starred = \Flagged
This is an excerpt from the imapsync output log done with the
--automap option. This listing is located just before the
folders listings.
This mapping can be changed by configuration in the Gmail
web interface so the result may vary.
=======================================================================
Q. Can I transfer Gmail "Chat" messages with imapsync?
R. No.
Even if you think you can because the Chat folder is seen in IMAP,

23
FAQ.d/FAQ.IceWarp.txt Normal file
View File

@ -0,0 +1,23 @@
#!/bin/cat
$Id: FAQ.IceWarp.txt,v 1.1 2020/02/25 10:49:07 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.IceWarp.txt
=======================================================================
Imapsync tips for IceWarp accounts.
=======================================================================
=======================================================================
Q. How to migrate from or to IceWarp?
R. Use:
imapsync ... --noabletosearch
The option --noabletosearch is needed only if options --maxage or --minage
are used. I don't know which --search commands could work, maybe none.

37
FAQ.d/FAQ.MailEnable.txt Normal file
View File

@ -0,0 +1,37 @@
#!/bin/cat
$Id: FAQ.MailEnable.txt,v 1.2 2020/07/07 10:45:56 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.MailEnable.txt
=======================================================================
Imapsync tips for MailEnable imap server
=======================================================================
======================================================================
Q: Syncing to MailEnable 10.31 and after
R: No special imapsync option is required, default is ok.
======================================================================
Q: Syncing from MailEnable 1.75
R: Use options:
imapsync ... --sep1 "/" --prefix1 ""
======================================================================
Q: Syncing from MailEnable 2.2
R: Use options:
imapsync ... --sep1 "." --prefix1 ""
======================================================================
Q: Syncing to MailEnable until MailEnable release 10.31 not included
R: Use options:
imapsync ... --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
======================================================================

30
FAQ.d/FAQ.Massive.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Massive.txt,v 1.30 2019/07/24 12:01:41 gilles Exp gilles $
$Id: FAQ.Massive.txt,v 1.31 2020/10/15 01:53:52 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -67,8 +67,26 @@ R2. To estimate the presync period, consider the mean imapsync transfer
Double the time because the best scenario never happens.
Triple the time because, well, real world is like that.
R3. To estimate the last period, just rerun a complete presync,
ie, resync all the mailboxes, the final sync should take
R3. An other way to better evaluate the end of the presync period can
be based on your actual data. Just apply a simple rule of three
on the mailboxes already migrated to estimate the global end.
If it took X hours to finish Y% of the mailboxes, then it
will take 100*X/Y hours to finish 100% of the mailboxes.
Following the same idea but using matematical garbage,
the ETA can be estimated like this:
t_0 = time of global start (the start of the first presync)
t_now = time of now.
Nb_total = total number of mailboxes to be migrated.
Nb_now = number of mailboxes already migrated.
then
ETA = t_end = (t_now - t_0) * (Nb_total / Nb_now) + t_0
R4. To estimate the last period, the final sync, just rerun a complete
presync, ie, resync all the mailboxes, the final sync should take
the same amount of time.
=======================================================================
@ -178,7 +196,7 @@ Here comes some measurements.
dstat -n -N eth0 6 # Linux only (in 2018)
Another excellent tool to measure the network trafic is iftop.
The following command will monitor imap and imaps connections
The following command will monitor imap and imaps connexions
on interfce eth0, only them, and sum them up:
iftop -i eth0 -f 'port imap or port imaps' -B # Linux
@ -250,9 +268,9 @@ Possible bottlenecks:
- Bandwidth.
Usually available bandwidth is NOT a bottleneck.
Meanwhile, it can be a bottleneck on small Internet connections.
Meanwhile, it can be a bottleneck on small Internet connexions.
Imapsync downloads messages from host1 and upload messages to host2,
consider this in case the connections are asymmetric.
consider this in case the connexion are asymmetric.
- I/O on disks.
I/O are a classical bottleneck, almost always forgotten.

73
FAQ.d/FAQ.Messages_Selection.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Messages_Selection.txt,v 1.15 2019/07/24 12:02:11 gilles Exp gilles $
$Id: FAQ.Messages_Selection.txt,v 1.17 2021/01/23 13:01:00 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -33,7 +33,53 @@ Q. What are the selection criteria available with --search option?
=======================================================================
Q. What messages imapsync syncs by default?
R. By default, Imapsync syncs all messages, except duplicates.
R1. By default, Imapsync syncs all messages, except duplicates.
R2. You can alter the default all messages selection by using one or
more of the following options:
Extract from the README https://imapsync.lamiral.info/README
OPTIONS/message selection
--maxsize int : Skip messages larger (or equal) than int bytes
--minsize int : Skip messages smaller (or equal) than int bytes
--maxage int : Skip messages older than int days.
final stats (skipped) don't count older messages
see also --minage
--minage int : Skip messages newer than int days.
final stats (skipped) don't count newer messages
You can do (+ zone are the messages selected):
past|----maxage+++++++++++++++>now
past|+++++++++++++++minage---->now
past|----maxage+++++minage---->now (intersection)
past|++++minage-----maxage++++>now (union)
--search str : Selects only messages returned by this IMAP SEARCH
command. Applied on both sides.
For a complete set of what can be search see
https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
--search1 str : Same as --search but for selecting host1 messages only.
--search2 str : Same as --search but for selecting host2 messages only.
So --search CRIT equals --search1 CRIT --search2 CRIT
--maxlinelength int : skip messages with a line length longer than int bytes.
RFC 2822 says it must be no more than 1000 bytes but
real life servers and email clients do more.
--useheader str : Use this header to compare messages on both sides.
Ex: Message-ID or Subject or Date.
--useheader str and this one, etc.
--usecache : Use cache to speed up next syncs. Not set by default.
--nousecache : Do not use cache. Caveat: --useuid --nousecache creates
duplicates on multiple runs.
--useuid : Use UIDs instead of headers as a criterion to recognize
messages. Option --usecache is then implied unless
--nousecache is used.
=======================================================================
Q. Is there a way we can specify a date range to sync emails?
@ -70,21 +116,27 @@ or
Q. Is there a way we can specify an age to sync emails?
If yes, can you please share some examples?
R. Yes, with the --maxage or the --minage option.
E.1 Sync only messages less than 2 days old:
Have in mind that the options --maxage and --minage can NOT be
combined successfully with the --search command. If the --search
option is present on the command line then --maxage and --minage
are ignored.
Example.1 Sync only messages less than 2 days old:
imapsync ... --maxage 2
E.2 Sync only messages more than 2 days old:
Example.2 Sync only messages more than 2 days old:
imapsync ... --minage 2
E.3 Sync only messages more than 30 days old and less than 365 days old:
Example.3 Sync only messages more than 30 days old and less than 365 days old:
imapsync ... --minage 30 --maxage 365
E.4 Sync only messages less than 30 days old or more than 365 days old:
Example.4 Sync only messages less than 30 days old or more than 365 days old:
imapsync ... --maxage 30 --minage 365
@ -101,9 +153,9 @@ Full explanation:
past|----maxage+++++minage---->now (intersection)
past|++++minage-----maxage++++>now (union)
C.1 By default,
Caveat.1 By default,
option --maxage is implemented as a --search SENTSINCE
Option --maxage is implemented as a --search SENTSINCE
RFC 3501 says: SENTSINCE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within or later than the specified date.
@ -151,12 +203,17 @@ Q. Can I migrate only mails with attachments?
R. Use:
imapsync ... --search "HEADER Content-Disposition attachment"
or
imapsync ... --search "HEADER Content-Type multipart/mixed"
or more generally:
imapsync ... --search "OR HEADER Content-Disposition attachment HEADER Content-Type multipart/mixed"
The last one doesn't work on all imap servers, not on Gmail at least (tested 2020/06/16)
=======================================================================
Q. How can I move messages marked \Deleted from all folders to

32
FAQ.d/FAQ.Messages_Too_Big.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Messages_Too_Big.txt,v 1.3 2019/06/18 16:37:39 gilles Exp gilles $
$Id: FAQ.Messages_Too_Big.txt,v 1.4 2020/10/21 09:21:48 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -11,6 +11,34 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt
Questions answered in this FAQ are:
Q. What does imapsync with big messages?
Q. What can I do to transfer messages bigger than what allows the
imap destination server?
Now the questions again with their answers.
=======================================================================
Q. What does imapsync with big messages?
R. By default, imapsync checks if the host2 imap server announces a
message size limit with "APPENDLIMIT=xxxx" in its CAPABILITY response.
Then imapsync takes this value to set automatically the --maxsize
option, unless --maxsize is already set on the command line
and has a smaller value than the APPENDLIMIT value.
If the host2 imap server doesn't announce any message size
limit then imapsync syncs messages no matter their sizes.
When a message is copied to the destination server, the destination
server accepts it or not but it does it as a whole, it takes the
full message or nothing. Nothing triggers an error in imapsync.
* Gmail announces APPENDLIMIT=35651584 (October 2020).
* Office365 announces nothing.
* Dovecot announces nothing.
=======================================================================
Q. What can I do to transfer messages bigger than what allows the
imap destination server?
@ -22,7 +50,7 @@ R1. A solution is to truncate the message to the maximum size allowed
on the software tool reading the email message.
For example,
Gmail currently limits messages to 35651584 bytes (May 2019),
Gmail currently limits messages to 35651584 bytes (October 2020),
Starting with imapsync release 1.938 option --truncmess xxxx truncates
messages bigger than the given size xxxx.

17
FAQ.d/FAQ.Migration_Plan.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Migration_Plan.txt,v 1.6 2019/04/10 12:05:31 gilles Exp gilles $
$Id: FAQ.Migration_Plan.txt,v 1.7 2021/03/02 22:31:36 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -41,12 +41,15 @@ Classical scenario, credentials triplets are different on both sides
follow this scenario.
* Presync all the mailboxes from the old server host1 to the new
server host2. Presyncs can usefully be done with --delete2 in
order to get an exact sync. But never use the option --delete2 once
users have started to play with their new account on host2, their
play will be lost on the next presync, or when the MX is changed,
since INBOX will start to receive new messages that are not on
host1.
server host2. If the imap server names are going to change their IP
resolution then don't use those names, use names that will always
match the same imap servers, or use their IP addresses.
Presyncs can usefully be done with --delete2 in order to get an
exact sync. But never use the option --delete2 once users have
started to play with their new account on host2, their play will be
lost on the next presync, or when the MX is changed, since INBOX
will start to receive new messages that are not on host1.
* Decide a migration day/hour.

510
FAQ.d/FAQ.Office365.txt Normal file
View File

@ -0,0 +1,510 @@
$Id: FAQ.Office365.txt,v 1.20 2021/03/17 20:42:35 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Office365.txt
=======================================================================
Imapsync tips for Office365.
=======================================================================
Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Office365 accounts?
Q. Does IMAPSync support OAUTH2 authentication in Office 365?
Q. How to sync from Office365 to XXX?
Q. How to sync from XXX to Office365
Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that?
Q. The imap connection to Office365 is not working very efficiently,
is there a solution to fix that?
Q. Office365 throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
Q. The sync fails with many "Trying command when NOT connected!".
What can I do?
Q. How to see or migrate public folders on Office365?
R. https://docs.microsoft.com/en-us/exchange/collaboration/public-folders/migrate-to-exchange-online?view=exchserver-2019
https://www.exchangesavvy.com/moving-your-public-folders-to-office-365-what-you-need-to-know/
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
Q. Office365 users complain that a folder named "Files" contains
messages with no sender.
Q. From XXX to Office365, read receipts are all resent again after sync.
Even for old messages. How can I fix that?
Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
Q. From XXX Office365 I get this error message sometimes:
"BAD Command Argument Error 11". What does it mean?
Q. From XXX to Office365 the flag Flagged does not seem to be well synced.
What can I do?
Q. How to migrate from or to Office 365 with an admin/authuser account?
Q. Couldn't create folder [trash] "Mailbox already exists".
Now the questions again with their answers.
=======================================================================
Q. Can I use imapsync to transfer from or to Office365 accounts?
R. Yes. But IMAP access to Office365 account is not always allowed by default
so it has to be allowed in the server configuration part.
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
=======================================================================
Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
R. No. But I plan to add it during the year 2021.
=======================================================================
Q. How to sync from Office365 to XXX?
R. On Windows, use:
imapsync.exe ... --office1
On Unix, use:
imapsync ... --office1
Option --office1 is like (release 1.970):
imapsync ... --host1 outlook.office365.com \
--ssl1 \
--exclude "^Files$"
=======================================================================
Q. How to sync from XXX to Office365
R0. IMAP is not enabled by default on Office365, see how to enable it:
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
If IMAP cannot be enabled or if it doesn't work well, you can try DavMail
(http://davmail.sourceforge.net/) which acts as an IMAP gateway
through Outlook Web Access.
R. Here is a command-line resume that solves most encountered issues when
migrating to Office365.
imapsync ... --office2
which is equivalent to (in imapsync release 1.870):
imapsync ... \
--host2 outlook.office365.com \
--ssl2 \
--maxsize 45000000 \
--maxmessagespersecond 4 \
--disarmreadreceipts \
--regexmess "s,(.{10239}),$1\r\n,g" \
--f1f2 "Files=Files_renamed_by_imapsync"
On Linux, you can also try the "reformime" command
that can be used like:
imapsync ... --maxlinelengthcmd "reformime -r7"
To get reformime, install the "maildrop" package.
On Linux again, there is a good Python script in the tarball that can
fix several things that Exchange or O365 have issues with.
Use it like this:
./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
It often does some miracles on messages.
=======================================================================
Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
R0. Imapsync doesn't support OAUTH2 authentication for Office365 accounts yet.
R1. Disable double-step authentication, also known as 2-factor,
2-step authentication on the Azure/Active Directory portal.
R2. Also, check a license is assigned to that account in Office365.
R3. Microsoft introduced something called "security defaults" which is enabled
by default for new tenants. One of the rules blocks IMAP access as of imapsync.
The funny thing is that you can't disable single rules of this security
package without buying additional licenses.
Switching the whole thing off allows the login.0
Also, disable double-step authentication on the Azure/Active Directory portal.
See here:
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
Thanks to Stephan Buhre for this R3 answer.
R4. Are there special characters in the password?
https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
R5. Triple check the hostname then. Try all of these:
* outlook.office365.com
* imap-mail.outlook.com
* imap.outlook.com
=======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that?
R. Office365 supports send/receive max message sizes of up to 150MB
but you need to make changes in your tenant(s) to support it.
The following PowerShell command will increase the message sizes that
can be sent/received. The trick in getting IMAPSync to work is to
apply these settings to the accounts performing the migration,
NOT the accounts associated with the target mailbox (assuming you're
using service accounts to perform transfers on behalf of users).
Set-mailbox -Identity $UPN -MaxReceiveSize 150mb -MaxSendSize 150mb
e.g.
Set-mailbox -Identity "migrationaccount@testtenant.onmicrosoft.com" -MaxReceiveSize 150mb -MaxSendSize 150mb
We're transferring data between Office 365 tenants so we set these
values on the migration accounts in the source and target tenants.
Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto
for this FAQ item.
=======================================================================
Q. The imap connection to Office365 is not working very efficiently,
is there a solution to fix that?
R. Yes. Try DavMail
http://davmail.sourceforge.net/
I don't use it myself but a user, Yannick Palanque, reported great results using it.
=======================================================================
Q. The sync fails with many "Trying command when NOT connected!".
What can I do?
R. The --debugimap option can show you more details, especially some
message like:
"BAD Request is throttled. Suggested Backoff Time: 178755 milliseconds"
In that case, see the next faq item and its fixes.
=======================================================================
Q. Office365 throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
R. Office365 has throttle mechanisms to limit any huge usage.
Sometimes imapsync transfers are too stressful for servers.
The following message
"Request is throttled. Suggested Backoff Time: 299961 milliseconds"
comes from the imap Office365 server, imapsync just reports it
before being disconnected from it.
To solve the throttles issues from 0365, there are two solutions
at least:
R1. Call Microsoft Office365 and ask them to remove the limits on your
mailboxes. That's not a joke, they do it for 90 days usually,
sometimes only after you reach the second technician you call,
the first one usually be not enough competent to understand
what you're talking about (I would be glad to remove this bad
fact).
R2. Play with options --maxbytespersecond or --maxmessagespersecond
or --exitwhenover
imapsync ... --maxbytespersecond 100_000
imapsync ... --maxmessagespersecond 2
imapsync ... --exitwhenover 1_000_000_000
I don't know the upper value that avoids the default throttling from 0365
and I guess it changes over time.
=======================================================================
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
R. The folder Files is a standard folder in Office365.
It should not be synced in IMAP. See the next question.
=======================================================================
Q. Office365 users complain that a folder named "Files" contains
messages with no sender.
R0. To fix this, add --exclude Files
imapsync ... --exclude Files
If you use --office1 then imapsync will add this exclusion
automatically like using the option:
imapsync ... --exclude "^Files$"
If you use --office2 then imapsync will add a renaming of any
"Files" folder on host1, like using the option:
imapsync ... --f1f2 "Files=Files_renamed_by_imapsync"
The host2 account ends up with a folder named
"Files_renamed_by_imapsync", but no complaining.
R1. This folder "Files" seems to be a standard folder in Exchange Online,
but it is not. This folder contains all attachments in every email
that is in the mailbox but without any headers.
This causes some confusion for users as these appear in their search
results as duplicate lines but without the sender details or even
the message body.
This folder seems to be usually hidden so IMAP clients can’t see it,
but for some reason sometimes it becomes visible.
R2. It looks to be a common problem with Exchange Online.
I’m not sure what causes the folder to appear.
More info here:
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
(This question/answer is taken quasi verbatim from Perttu Aaltonen)
=======================================================================
Q. From XXX to Office365, read receipts are all resent again after sync.
Even for old messages. How can I fix that?
R. Imapsync can remove the header containing this read-receipt request.
On Unix or Windows use:
imapsync ... --disarmreadreceipts
Since read receipts should be sent for unseen messages that will go
to a seen state after the migration, you could be strict and apply
the regex only to seen messages.
Selecting seen message can be done with:
imapsync ... --search1 "SEEN" --disarmreadreceipts
A second run has to be run without the --disarmreadreceipts
for unseen messages:
imapsync ... --search1 "UNSEEN"
If fact --disarmreadreceipts is just an option equivalent to:
--regexmess 's{\A(.*?(?! ^$))^Disposition-Notification-To:(.*?)$}{$1X-Disposition-Notification-To:$2}igxms'
That regex changes the header Disposition-Notification-To. It prefixes
it with an X- so that it becomes inactive.
Disposition-Notification-To: blabla
becomes
X-Disposition-Notification-To: blabla
Thanks to David Karnowski for pointing and solving this issue.
=======================================================================
Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
R1. "SSL read or write error" happens sometimes, it isn't related to
imapsync directly but to the ssl underlying library when communicating
with Exchange in TLS/SSL encrypted mode.
Next runs should put the sync further, so rerun the syncs
until it is well completed.
R2. Another solution is to remove --tls or --ssl options for Exchange
and accept clear text syncs.
R3. See also the FAQ FAQ.SSL_errors.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
=======================================================================
Q. From XXX Office365 I get this error message sometimes:
"BAD Command Argument Error 11". What does it mean?
R. This error message comes from the Office365 IMAP server when it
encounters any problem. Most of the time it is one of the following:
* Some messages are bigger than the size limit.
45 MB by default on Office365.
I don't know if it can be upped by configuration for Office365.
If you can't fix this limit on Office365 then use the option
--maxsize 45000000 for 45 MB to tell imapsync to skip those messages.
imapsync ... --maxsize 45000000 # 45 MB for Office365
* Quota reached. The whole account is full.
It can be upped by configuration.
* You use --synclabels --resynclabels from a previous command line
related to syncing from Gmail to Gmail. Remove them.
* Some messages have some lines too long. Use option --maxlinelength
to skip messages whose max line length is over some bytes.
--maxlinelength 1000 is an RFC2822 must but most servers support
higher values. Exchange supports 9900 characters line length:
imapsync ... --maxlinelength 9900
Office365 supports 10500 characters line length:
imapsync ... --maxlinelength 10500
In case you prefer fixing messages with long lines the hard way,
instead of skipping them with --maxlinelength 9900, just use:
On Windows
imapsync ... --regexmess "s,(.{9900}),$1\r\n,g"
On Unix
imapsync ... --regexmess 's,(.{9900}),$1\r\n,g'
Have also in mind that Exchange closes the connection after 10 errors
encountered so you might also see "BYE Connection closed" errors from
Exchange, which means Exchange leaves the session and says goodbye,
come back later. Redo some sync then.
On Linux, there is a good Python script in the tarball that can
fix several things that Office365 has issues with.
Use it like this:
./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
It often does some miracles on messages.
It's called fix_email_for_exchange because Office365 is Exchange,
or at least started to be an Exchange server with the same issues.
=======================================================================
Q. From XXX to Office365 the flag Flagged does not seem to be well synced.
What can I do?
R. Use the following trick. Run imapsync twice, one with --regexflag
and one without, like this:
1) imapsync ... --regexflag "s/\\Flagged//g"
2) imapsync ...
You can add --debugflags if you want to see what imapsync gets and
does in detail with flags.
The magic of this trick is on ignoring the \Flagged flag on the first
sync and setting it on the second sync, with STORE instead of APPEND.
This Exchange bug seems that Exchange gets and sets well the Flagged
flag with APPEND in IMAP but then it forgets it with other protocols;
With STORE it sets and gets the "\Flagged" flag everywhere.
Thanks to Dave Murray and Simon Savva for reporting and solving
this issue.
======================================================================
Q. How to migrate from or to Office 365 with an admin/authuser account?
Note from Yago Torres Fernandez:
(a working command using admin/authuser on host2 Office 365)
imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com ^
--password2 XXXX --ssl2 ^
but previously in Office365, you must do something like the following, using Powershell:
Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all
Note from Betsy Lawlor: You can use global modern authentication with two factor
on Exchange Online (M365) but you must have "AllowBasicAuthImap" on the admin account you are using to migrate the mail.
Remark: PLAIN authentication is the only way to go with --authuser1 for now.
So don't use --authmech1 SOMETHING with --authuser1 admin_user,
it will not work.
Same behavior with the --authuser2 option.
Note from Rafael Alvarez Ballesteros:
When you get an Office365 license you will receive an admin user to handle
your licenses and products like admin@yourcompanyname.onmicrosoft.com.
This account is the administrator account; some weeks or months ago Microsoft
has decided you need to use two-factor authentication by default,
so if two factors authentication is enabled you will no be able to sync
the mail (it will not connect to host2).
OWA (I think this is the two-factor authentication) needs to be disabled
globally and enabled individually on the users you want to.
One account can have the rights to access other mailboxes no matter if admin or
any other account but needs to have OWA disabled to be able to connect
to the office365 server.
Note from Martin Paulucci:
I had to remove the domain part for the user
but not for the admin. Example:
imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated
See also:
http://www.linux-france.org/prj/imapsync_list/msg02203.html
Subject: RE: [imapsync] Office 365 - 'Master User'?
Date: Mon, 1 Jun 2015 17:53:54 +0000
=======================================================================
Q. Couldn't create folder [trash] "Mailbox already exists".
R. Some servers take care of character cases in folder names,
some servers do not, like Exchange. Since not respecting the case can merge
two different folders into one then imapsync respects the case.
For example, if a host1 server has a folder name called "trash"
and the host2 server already has a folder "Trash" or "TRASH"
then imapsync will try to create the folder "trash" on host2
because trash and Trash are different strings. But if host2
does not respect character case it will consider folder "trash"
already exists and will say it. That's the error message reported
by imapsync: "Mailbox already exists". This message comes from the server.
The folder creation fails but messages are well transferred in
so take a look at this warning, understand why it happens
and it should be fine most of the time.
To avoid this warning use --regextrans2 to map the folder names
imapsync ... --regextrans2 "s/^trash$/Trash/"
If there are two folders Trash and trash on host1 then both
will be merged into only one Trash folder on host2.
In case of the option --delete2 is used the regextans2 above becomes
mandatory, otherwise imapsync will sync messages from the
first Trash and then delete them when syncing trash.
If you want to avoid merging folders that are considered different
on host1 but are considered the same on destination host2 because of
case sensitivities and insensitivities, use --nomixfolders
=======================================================================
=======================================================================

36
FAQ.d/FAQ.Old_Style_Web_Design.txt
View File

@ -1,20 +1,14 @@
#!/bin/cat
$Id: FAQ.Old_Style_Web_Design.txt,v 1.4 2019/11/25 13:08:31 gilles Exp gilles $
$Id: FAQ.Old_Style_Web_Design.txt,v 1.9 2020/12/02 21:38:13 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Old_Style_Web_Design.txt
=======================================================================
Imapsync tips for Gmail accounts.
Imapsync and its ugly website style
=======================================================================
Questions answered in this FAQ are:
Q. Why the imapsync website looks so old fashion?
Now the questions again with their answers.
=======================================================================
Q. Why the imapsync website looks so old fashion?
@ -29,28 +23,44 @@ Here are several reasons it stays like that for now:
* I like to do things myself
* I like static web
* I am lazy
* I like static web.
Although you could argue that I do use some dynamic things, like SSI.
That's true, I use Server Side Include, aka SSI, denoted by the ".shtml"
extention of some files, instead of the classical ".html" extention.
It's because I like dynamic web when the dynamic brings something that
can't be done easily with static files.
* I'm proud to be w3c compliant
https://validator.w3.org/check?uri=https%3A%2F%2Fimapsync.lamiral.info
Very very few sites are w3c compliant, it's because all CMS frameworks aren't.
Very very few sites are w3c compliant, it's because nearly all CMS frameworks aren't.
Only true crazy people are w3c compliant seekers. I'm one of them.
If you know a CMS w3c compliant, tell me, it will be such a crazy news.
If you know any web site w3c compliant, tell me, it will be crazy news as well.
Update: https://contao.org/ is a CMS and it is w3c compliant! This is astonishing!
(Thanks to Markus Rupprecht for this crazy input!)
* I prefer to spend more time on the product itself than the showcase
* The product is a command line, a fancy website to sell a command line
makes more disappointed buyers
I agree I could do some sort of A/B experiment and see whether during
a month I sell more imapsync on a brand new style B site than
the current A one.
Besides, don't misunderstand buyers. The surprising thing about complainers
is that all are web designers, none is a user not in the webdesign branch.
is that all are web designers, none of them is a user not in the webdesign branch.
I even received some phone call from a buyer saying: "I saw your website
and I told to myself: this can't be a fake one, the product must be a good one".
Another folk like me:
http://keyhut.com/pos.htm
(well, not w3c compliant, since only mad people try to be w3c compliant)
(well, not w3c compliant, he is not that mad)
So maybe one day...

115
FAQ.d/FAQ.OnlineUI.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.OnlineUI.txt,v 1.20 2019/12/06 14:51:03 gilles Exp gilles $
$Id: FAQ.OnlineUI.txt,v 1.27 2021/04/24 16:57:50 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -7,7 +7,6 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.OnlineUI.txt
=====================================================================
Imapsync tips about the online visual user interfaces
https://i005.lamiral.info/X/
https://imapsync.lamiral.info/X/
=====================================================================
@ -15,9 +14,18 @@ Questions answered in this FAQ are:
Q. How secure is the online visual user interface /X?
Q. Will I have any issues with browser timing out? What happens
Q. Does the online service store any sensitive information
like my passwords?
Q. Shall I have issues when the browser times out? What happens
if the browser connection is closed for whatever reason?
Q. Shall I have issues when the webserver times out? What happens
if the webserver closes the connection for whatever reason?
Q. The sync stalls at the beginning, just after a line like:
"Host1: xxx says it has CAPABILITY for AUTHENTICATE LOGIN"
What is the problem?
Now the questions again with their answers.
@ -30,42 +38,41 @@ R0. Well, I don't know if asking the provider whether his online
R1. Some figures
Date of this report: 6 December 2019.
Date of this report: 22 April 2021.
The online imapsync service /X started 9 January 2017
(1061 days of service).
(1564 days of service).
In average, /X has 50 users per day lunching in mean 6
On average, /X has 51 users per day lunching in mean 6
different migrations, from just one launch to many (hundreds).
The total volume /X transferred is around 101 TiB in more
than 219 thousands email imap migrations,
340 millions email messages.
The total volume /X transferred is around 210 TiB from nearly
450 000 imap account migrations and 700 million email messages.
R2. Pros & Cons
The online imapsync service /X runs on https only, with a
The online imapsync service /X runs on HTTPS only, with a
letsencrypt certificate, a certificate overall rated "A+" at
https://www.ssllabs.com/ssltest/analyze.html?d=i005.lamiral.info
https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
Because of the https usage, what the users enter in their browser,
Because of the HTTPS usage, what the users enter in their browser,
the imap logins and passwords, can't be eavesdropped on the network.
Imapsync itself takes care about encryption for the imap sessions,
Imapsync itself takes care of encryption for the imap sessions,
if possible: It tries SSL first on port 993, then TLS on port 143
if the servers announces TLS, then no encryption at all.
if the servers announce TLS, then no encryption at all.
Concerning encryption, what is done with the source imap server host1
is independent of what is done with the destination imap server host2.
At the date of 6 December 2019, there is no security problem
On the date of 22 April 2021, there is no security problem
detected or reported to me (Gilles LAMIRAL), so far.
Feel free to attack the service and feel free to report any
hole encountered. Have in mind I can watch what you try
from the server side and take measure if the service suffers from
from the server-side and take measures if the service suffers from
your acts.
As the owner of the service, it could have been 219 000 pairs of
credentials collected and nearly 101 terabytes of email messages.
As the owner of the service, it could have been 450 000 pairs of
credentials collected and nearly 210 terabytes of email messages.
I haven't kept them but I can't prove I haven't. It's just trust,
like nearly every online service in the universe.
@ -77,26 +84,43 @@ This default behavior is chosen like this because users of /X
want their emails transferred, instead of being not transferred
because of an incompetent imap server sysadmin.
I admint that this part, checking imap ssl/tls certificates,
I admit that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known
certificates directly in imapsync.
If the imap servers don't honor ssl nor tls, then logins, passwords
and everything will go clear text during the imap transfers.
That's not good at all but what "comforts" me is that if the
imap servers do only clear text transfers, then it's also true
for all imap sessions the accounts' owner encounters,
imapsync is just one of them.
That's not good at all but what "comforts" me is that if an
imap server does only clear text transfers, then it's also true
for all the imap sessions the account owner encounters,
imapsync is just one of them; the imap servers are then always
unsecure for any imap software client.
Last point, who could be sure that no cracker cracked the online
hosts and that he isn't currently sniffing the credentials?
No one, I'm not sure myself, even if I do take care of that
possibility. So changing the imap accounts passwords after
a sync is a safe and recommended practice!
the sync is a safe and recommended practice!
=====================================================================
Q. Will I have any issues with browser timing out? What happens
Q. Does the online service store any sensitive information
like my passwords?
No.
The online passwords are kept by your browser. The goal is
that the next time you run a sync from your browser, all the
parameters are there without having to re-enter them.
The save is done when you click on the "Sync or resync" green
button. So, if you don't want your browser to keep your good password,
enter a wrong one or empty the password field and click
on the "Sync or resync" green button, the wrong password
will be saved in your browser and the sync won't be done,
since a good login is mandatory to do anything on your mailboxes.
=====================================================================
Q. Shall I have issues with the browser timing out? What happens
if the browser connection is closed for whatever reason?
R. It stops the imapsync process, ie, the sync is ended right away.
@ -105,21 +129,21 @@ Further comments on this behavior.
When using the /X interface there are three connections.
One connection is the Browser/WebServer connection,
the two others connections are the WebServer/ImapServers
the two other connections are the WebServer/ImapServers
connections (imapsync stuff).
If the Browser/WebServer connection is timeout or ended,
If the Browser/WebServer connection timeouts or ends,
the imapsync sync is also ended immediately by the remote
Apache https server. Technically, Apache sends a TERM signal
Apache HTTPS server. Technically, Apache sends a TERM signal
to the imapsync process, then wait some seconds before
sending a KILL signal if it is still alive.
You can relaunch a sync again with "Sync!" button, at any time.
You can relaunch the sync with the "Sync!" button, at any time.
If the "Sync!" button is gray/inactive then just reload
the page (F5 or similar), and reenter the credentials.
If the interface tells you that a sync is already going on,
it may be that a sync is running from another browser or place.
If the interface tells you that the sync is already going on,
it may be that the sync is running from another browser or place.
You can stop this sync with the "Abort!" button from any /X
tab/window, even from another browser or place. To be able
to abort with success, you have to give the same account
@ -127,10 +151,35 @@ parameters, same credentials, or imapsync will ignore the demand.
In other words, you can try safely to launch several parallel
runs between the same mailboxes. Open a new tab/windows with /X,
and start the exact same sync. It's safe, the /X will say, if any, that
there is already a current sync running on them and it will present
and start the same sync. It's safe, the /X page will say that
there is already one sync running and it will present
the logfile running the sync like a "tail -f" command (isn't that magic?).
=====================================================================
Q. Shall I have issues when the webserver times out? What happens
if the webserver closes the connection for whatever reason?
R. If the webserver closes the connection then usually it kills
imapsync process and the imap connections as well.
To fix this issue, see the document INSTALL.OnlineUI.txt
and search for "Timeout" in it.
Tip: If the connections close for huge folders and always after
300 seconds or 60 seconds then it looks like a webserver timeout.
Apache 2.2 timeout is 300 seconds.
Apache 2.4 timeout is 60 seconds.
=====================================================================
Q. The sync stalls at the beginning, just after a line like:
"Host1: xxx says it has CAPABILITY for AUTHENTICATE LOGIN"
What is the problem?
R. I've seen this issue on /X with the imap server
CommuniGate Pro IMAP Server 6.0.11
The issue looks related to special characters in the password.
Solution for now: change the password, keep only
standard (ASCII) alphanumeric characters ABC-YZ abc-yz 012-89.
=====================================================================
=====================================================================

38
FAQ.d/FAQ.Passwords_on_Mac.txt Normal file
View File

@ -0,0 +1,38 @@
#!/bin/cat
$Id: FAQ.Passwords_on_Mac.txt,v 1.1 2020/02/12 11:25:48 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
=======================================================================
Imapsync issues with passwords on Mac OS X.
=======================================================================
=======================================================================
Q. On Mac, some passwords contain some weird *()",;&~ characters.
Login fails.
R1. Enclose the password within single-quotes in the imapsync
command line:
imapsync ... --password1 'passw*()",;&rd~'
CAVEAT: Be careful to use quotes and double-quotes characters encoded
from the ASCII table. How to be sure of that?
Mac OS X has a "use smart quotes" option. This is particularly handy
when typesetting and the system automatically substitutes opening and
closing quotes. It isn't handy when editing the imapsync command line.
Disable "use smart quotes and dashes" in the Keyboard configuration,
located in the "Text" tab. It will fix this unfortunate character
transformation. You can put it back later.
R2. Change the password in order to keep only f...ing normal characters,
ie, alphanumerics like aAzZ-_09, and minus and underscore.
Make it long and random if strong security is your concern.
=======================================================================
=======================================================================

18
FAQ.d/FAQ.Passwords_on_Unix.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Passwords_on_Unix.txt,v 1.6 2019/04/02 17:19:21 gilles Exp gilles $
$Id: FAQ.Passwords_on_Unix.txt,v 1.7 2020/12/04 11:46:54 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -12,19 +12,23 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
=======================================================================
Q. On Unix, some passwords contain some weird *(),;&~ characters.
Q. On Unix, some passwords contain some weird *()$,";&~ characters.
Login fails.
R1. Enclose the password within single-quotes in the imapsync
command line:
imapsync ... --password1 'passw*(),;&rd~'
imapsync ... --password1 'passw*()`"$,;&rd~'
R2. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
R2. Change the password to keep only f...ing normal characters.
Make it long and random if strong security is your concern.
R3. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
try also using double-quotes within single-quotes. It will enclose
the password within double-quotes in the imap LOGIN command:
imapsync ... --password1 '"passw*(),;&rd~"'
imapsync ... --password1 '"passw*()`$,;&rd~"'
Old Mail::IMAPClient 3.37 is a bad horse having this issue solved
by this double-quotes trick.
@ -32,9 +36,5 @@ fixed in 3.38 https://metacpan.org/changes/distribution/Mail-IMAPClient
version 3.38: Tue Feb 9 02:48:21 UTC 2016
- rt.cpan.org#110273: failure to quote password values
(regression introduced in 3.36 via fix for rt.cpan.org#100601)
R3. Change the password to keep only f...ing normal characters.
Make it long and random if strong security is your concern
=======================================================================
=======================================================================

38
FAQ.d/FAQ.Principles.txt
View File

@ -1,5 +1,5 @@
$Id: FAQ.Principles.txt,v 1.5 2019/09/05 13:33:46 gilles Exp gilles $
$Id: FAQ.Principles.txt,v 1.10 2021/04/18 13:00:00 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -13,28 +13,36 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Principles.txt
Questions answered in this FAQ are:
Q. What is the bandwidth used by imapsync?
Q. Give me some design ideas of imapsync
Now the questions again with their answers.
=======================================================================
Q. What is the bandwidth used by imapsync?
Q. Give me some design ideas of imapsync
R. From the host where imapsync runs, it opens two imap connections,
one with the source account at host1, one with the destination
account at host2.
R.
So, the global bandwidth used is twice the volume of the source
account, one volume to download the messages from host1,
one volume to upload those messages to host2.
* No configuration file
* Stateless
* Rsync in mind
* Reliable
* Robuste
* The user is not an expert, help him with:
* Good behavior by default.
* No harm by default.
* Prefer fixing an issue in the code than documenting the issue
* Inline help suggestions
If the host2 is already filled with the messages, imapsync doesn't
transfer them and then the volume transferred is small, just the IMAP
commands to identify the messages on both sides.
Consequences of those design ideas:
* Stateless:
* A sync can be done by another tool before or after imapsync, it is
safe as long as the other tools don't modify the messages' headers
"Message-Id:" and "Received:", or at least "Message-Id:".
* Ready to run in a container.
There is no local cache of the email messages, except when a
message is very big, it is temporarly saved locally.
=======================================================================

5
FAQ.d/FAQ.Release_Checklist.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Release_Checklist.txt,v 1.16 2020/01/04 09:35:36 gilles Exp gilles $
$Id: FAQ.Release_Checklist.txt,v 1.18 2020/01/06 10:47:02 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -32,13 +32,14 @@ Checklist before release a new release:
- /X verify under noscript with firefox
- /X verify under private with chrome
- Check the binaries imapsync.exe imapsync_64bit.exe with
- Check the binaries imapsync.exe imapsync_32bit.exe with
https://www.virustotal.com/
https://www.metadefender.com/
- make dist
- verify on Windows the zip extraction and simple execution of imapsync_example.bat
- verify binary version matches imapsync script version
- make publish
- verify VERSION is uploaded

134
FAQ.d/FAQ.SSL_errors.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.9 2019/12/11 15:38:37 gilles Exp gilles $
$Id: FAQ.SSL_errors.txt,v 1.14 2021/04/19 11:47:44 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -21,7 +21,11 @@ Q. What can I do to avoid those "SSL read/write errors"?
Q. SSL connect attempt failed SSL
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
or
SSL connect attempt failed SSL
SSL routines:ssl_choose_client_version:unsupported protocol
Q. How to see the certificate and identify problems in it?
Now the questions again with their answers.
@ -32,29 +36,29 @@ Q. What are the errors
DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
R1.Like they claim, those errors are SSL errors. SSL is not directly
done by imapsync but by an underlying Perl module called
IO::Socket::SSL. Those errors arise sometimes and sometimes
they form a serie that ends with imapsync auto-abortion.
Those errors happen with some hosts but not with others,
it's often Exchange or Office365. I don't know what exactly happens.
Those errors happen more often on Windows than on Linux.
R1. As they claim, those errors are SSL errors. SSL is not directly
done by imapsync but by an underlying Perl module called
IO::Socket::SSL. Those errors arise sometimes and sometimes
they form a series that ends with imapsync auto-abortion.
Those errors happen with some hosts but not with others,
it's often Exchange or Office365. I don't know what exactly happens.
Those errors happen more often on Windows than on Linux.
=======================================================================
Q. What can I do to avoid those "SSL read/write errors"?
R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
Those errors don't appear with recent releases, post 1.836
Those errors don't appear with recent releases, post 1.836
R1. Remove all ssl/tls encryption
imapsync ... --nossl1 --notls1 --nossl2 --notls2
R2. If you don't want to quit encryption, rerun imapsync until the
complete sync is over. Those errors are not at the same place
each time, so imapsync will sync remaining messages at each run
until none remains.
complete sync is over. Those errors are not at the same place
each time, so imapsync will sync the remaining messages at each run
until none remains.
R3. Run imapsync on a Linux machine, a VM is ok, there are less
SSL errors on Unix.
@ -69,12 +73,16 @@ R5. Set up a ssltunnel proxy to the host.
=======================================================================
Q. SSL connect attempt failed SSL
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
or
SSL connect attempt failed SSL
SSL routines:ssl_choose_client_version:unsupported protocol
R1. Use:
imapsync ... --sslargs1 'SSL_cipher_list=DEFAULT'
imapsync ... --sslargs1 SSL_cipher_list=DEFAULT
or
imapsync ... --sslargs2 'SSL_cipher_list=DEFAULT'
imapsync ... --sslargs2 SSL_cipher_list=DEFAULT
depending on where the error occurs, host1 or host2 or both.
@ -82,13 +90,27 @@ R2. If it doesn't work, I let you try other things,
I quote the "SSL_version" section of
https://metacpan.org/pod/IO::Socket::SSL (Module version: 2.066)
imapsync ... --sslargs1 SSL_cipher_list=DEFAULT
imapsync ... --sslargs1 SSL_version=SSLv2
imapsync ... --sslargs1 SSL_version=SSLv23
imapsync ... --sslargs1 SSL_version=SSLv3
imapsync ... --sslargs1 SSL_version=TLSv1
imapsync ... --sslargs1 SSL_version=TLSv1_1
imapsync ... --sslargs1 SSL_version=TLSv1_2
imapsync ... --sslargs1 SSL_version=TLSv1_3
SSLv2 and TLSv12 are just examples depending on your context
(--ssl1 or --tls1, and also the imap server encryption scheme)
Those examples are for host1. For host2, use --sslargs2 instead.
Feedback on what worked for you is welcome!
Feedback on what worked for you (and possibly hy) is welcome!
A loop to check every version and print the good ones:
for v in SSLv2 SSLv23 SSLv3 TLSv1 TLSv1_1 TLSv1_2 TLSv1_3; do
imapsync ... --sslargs1 SSL_version=$v && GOOD="$GOOD $v"
done
echo "$GOOD"
I reproduce below the documentation of the underlying Perl
module IO::Socket::SSL used by imapsync:
https://metacpan.org/pod/IO::Socket::SSL
...
@ -124,10 +146,78 @@ In this case setting the version
to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.
=======================================================================
Q. How to see the certificate and identify problems in it?
R. Use the command openssl like this:
echo | openssl s_client -crlf -connect imap.gmail.com:993
echo | openssl s_client -crlf -connect test1.lamiral.info:993
and examine carefully the content, the "verify return:" lines,
the chain. Sometimes, the server certificate is ok but not the whole
chain of certificates so the certification fails.
Here is an example.
One of the certificate is expired:
echo | openssl s_client -crlf -connect test1.lamiral.info:993
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = test1.lamiral.info
verify error:num=10:certificate has expired
notAfter=Apr 11 10:14:05 2021 GMT
verify return:1
depth=0 CN = test1.lamiral.info
notAfter=Apr 11 10:14:05 2021 GMT
verify return:1
---
Certificate chain
0 s:/CN=test1.lamiral.info
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKjCCBBKgAwIBAgISBHYZCE3qSTIlvq97HI5TpBeAMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAxMTExMDE0MDVaFw0yMTA0MTExMDE0MDVaMB0xGzAZBgNVBAMT
EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAMUTJVdrTl86nDI2yO6Vz5l1qxMMPqJylQcgi9vDHpwsnUq5HGPv+qZNhM69
...
After an complete server update ("apt update && apt upgrade"):
echo | openssl s_client -crlf -connect test1.lamiral.info:993
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = test1.lamiral.info
verify return:1
---
Certificate chain
0 s:/CN=test1.lamiral.info
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKTCCBBGgAwIBAgISBD4QN3cfB1JpTm75oVrkkAElMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAzMTIxODQxMTJaFw0yMTA2MTAxODQxMTJaMB0xGzAZBgNVBAMT
EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
...
=======================================================================
=======================================================================

21
FAQ.d/FAQ.Security.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: FAQ.Security.txt,v 1.19 2018/09/13 17:53:45 gilles Exp gilles $
# $Id: FAQ.Security.txt,v 1.22 2020/06/30 17:32:39 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -32,7 +32,7 @@ R1. The online UI does TLS/SSL imap connections if the imap
servers support TLS/SSL. If you are concerned by security then
using the .bat file or .sh on your computer should be more secure
since you can examine and secure it by yourself, no matter high is
your paranoïd spirit. The online UI security is mine, I am
your paranoid spirit. The online UI security is mine, I am
concerned by security, not to the upmost high level possible
but I won't give you direct access to the host to discover my level.
With a good guy spirit, feel free to try to break the online UI
@ -159,8 +159,8 @@ b OK 73 good day (Success)
The client part you have to type is
a LOGIN ...
b LOGOUT
while replacing ... by your credentials values.
while replacing ... by your credentials values,
other lines are the server responses.
=======================================================================
Q.How can I test an tls imap connection without imapsync?
@ -200,6 +200,19 @@ to help the ssl software verifying the server certificate.
The file ca-certificates.crt may be elsewhere on your system, even
named differently.
The imap server certificates are not checked for authenticity
(by default) because too many imap servers are crappy configured
regarding certified certificates.
This default behavior is chosen like this because users
want their emails transferred, instead of being not transferred
because of an incompetent imap server sysadmin.
I admint that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known
certificates directly in imapsync.
Drop me a note to encourage me.
=======================================================================
Q: How to have an imaps server?

43
FAQ.d/FAQ.SmarterMail.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.SmarterMail.txt,v 1.14 2019/10/15 14:57:19 gilles Exp gilles $
$Id: FAQ.SmarterMail.txt,v 1.15 2021/04/30 13:02:17 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -10,6 +10,39 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.SmarterMail.txt
Imapsync tips for SmarterMail. Specific issues and solutions.
=======================================================================
https://imapsync.lamiral.info/README
...
--prefix1 str : Remove prefix str to all destination folders,
usually "INBOX." or "INBOX/" or an empty string "".
imapsync guesses the prefix if host1 imap server
does not have NAMESPACE capability. So this option
should not be used most of the time.
--prefix2 str : Add prefix to all host2 folders. See --prefix1
--sep1 str : Host1 separator. This option should not be used
most of the time.
Imapsync gets the separator from the server itself,
by using NAMESPACE, or it tries to guess it
from the folders listing (it counts
characters / . \\ \ in folder names and choose the
more frequent, or finally / if nothing is found.
--sep2 str : Host2 separator. See --sep1
--noabletosearch : Makes --minage and --maxage options use the internal
dates given by a FETCH imap command instead of the
"Date:" header. Internal date is the arrival date
in the mailbox.
--noabletosearch equals --noabletosearch1 --noabletosearch2
--noabletosearch1 : Like --noabletosearch but for host1.
--noabletosearch2 : Like --noabletosearch but for host2.
--useheader str : Use this header to compare messages on both sides.
Example: Message-ID or Subject or Date.
--useheader str and this one, etc.
=======================================================================
Q. Synchronizing from SmarterMail to XXX
@ -17,12 +50,12 @@ Q. Synchronizing from SmarterMail to XXX
On Unix:
imapsync --host1 imap.d1.org --user1 joe --password1 secret1 \
--host2 imap.d2.org --user2 joe --password2 secret2 \
--sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch
--prefix1 "" --sep1 "/" --useheader Message-Id --noabletosearch
On Windows:
imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^
--host2 imap.d2.org --user2 joe --password2 secret2 ^
--sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch
--prefix1 "" --sep1 "/" --useheader Message-Id --noabletosearch
=======================================================================
@ -31,12 +64,12 @@ Q. Synchronizing from XXX to SmarterMail
On Unix:
imapsync --host1 imap.d1.org --user1 joe --password1 secret1 \
--host2 imap.d2.org --user2 joe --password2 secret2 \
--sep2 "/" --prefix2 "" --useheader Message-Id --noabletosearch
--prefix2 "" --sep2 "/" --useheader Message-Id --noabletosearch
On Windows:
imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^
--host2 imap.d2.org --user2 joe --password2 secret2 ^
--sep2 "/" --prefix2 "" --useheader Message-Id --noabletosearch
--prefix2 "" --sep2 "/" --useheader Message-Id --noabletosearch
=======================================================================

65
FAQ.d/FAQ.Two_Ways_Sync.txt Normal file
View File

@ -0,0 +1,65 @@
$Id: FAQ.Two_Ways_Sync.txt,v 1.4 2021/02/01 15:43:33 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Two_Ways_Sync.txt
=======================================================================
Two ways sync with Imapsync
=======================================================================
Questions answered in this FAQ are:
Q. Can Imapsync do a good "two ways" sync?
No. Why?
Now the questions again with their answers.
=======================================================================
Q. Can Imapsync do a good "two ways" sync?
No. Why?
R. Imapsync can't do good two ways syncs.
A good "two ways" sync is impossible with imapsync because imapsync is
stateless.
Each time imapsync runs, it considers messages and folders as if it
were the first time it encounters them. Imapsync looks at messages,
flags, and folders as they are now, not considering what they were
before. Imapsync has no memory outside the current running sync.
So now, why a stateless behavior cannot handle well a two ways sync
between an account A and an account B?
The problem arises with deletions, messages deletions, folders
deletions, or movings, messages movings across folders, folders
movings, and also folders renamings. Deletions and moves are ambiguous
changes when combined with creations on the opposite side.
For example, when a message is deleted from A by a user, imapsync
cannot know whether it is a message deleted from A that has to be
deleted in B (what the user did) or a missing message from B that has
to be copied to A.
But if you know the answer yourself, that missing messages on one side
A are deleted messages that have to be deleted on the other side then
run a sync with the --delete2 option from A to B.
If you know that the missing messages on A are missing messages from B
that has to be copied to A then run a sync from B to A.
If you know it's a mixed scenario then you are in trouble and so you
end up with a not very good "two ways" sync. I suggest avoiding
deletions in that case, which is the default imapsync behavior.
With a two ways sync, the mailbox user is very surprised and
disapointed when his actions (deletions, renamings, or movings) come
back.
=======================================================================
=======================================================================

43
FAQ.d/FAQ.Use_cache.txt
View File

@ -1,5 +1,5 @@
$Id: FAQ.Use_cache.txt,v 1.9 2019/09/13 10:43:06 gilles Exp gilles $
$Id: FAQ.Use_cache.txt,v 1.11 2021/04/22 13:20:18 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -7,7 +7,7 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
=======================================================================
Imapsync --usecache option
Imapsync --usecache or --useuid options
=======================================================================
Questions answered in this FAQ are:
@ -16,6 +16,9 @@ Q. What is --usecache good for?
Q. What is the technical principle of --usecache?
Q. On Windows, with --useuid and after a reboot or a new login, a new
run generates duplicates. What can I do to avoid and fix that?
Q. On Windows, with --useuid or --usecache a problem occurs with long
nested folder names. The error message is:
"No such file or directory; The filename or extension is too long"
@ -23,14 +26,14 @@ Q. On Windows, with --useuid or --usecache a problem occurs with long
Q. Inode problem with --usecache or --useuid on Linux
Now the questions again and their answers
Now the questions again and their answers.
=======================================================================
Q. What is --usecache good for?
R. The option --usecache is useful to speed up next syncs with --usecache.
The first sync with --usecache between two imap accounts is as low
as the first sync without --usecache.
as the first sync without --usecache, the next syncs are faster.
=======================================================================
Q. What is the technical principle of --usecache?
@ -49,6 +52,38 @@ R. Option --usecache creates a empty file per email message
where <tmpdir> is the temporary directory given by the --tmpdir option
or the one by default, /tmp on Unix, variable on Windows.
=======================================================================
Q. On Windows, with --useuid and after a reboot or a new login, a new
run generates duplicates. What can I do to avoid and fix that?
R. The cache is build in a temporary directory. Imapsync uses a system
call to get this temporary directory, it is not hardcoded and it
is system dependent, the call garanties a write access. Sometimes on
Windows this directory name changes upon reboot or for any other reason.
Anyway, the temporary directory used by imapsync can be set by the
option --tmpdir and that's the way to avoid this issue.
The new problem is that the temporary directory must be writable, which
is not a big deal to found.
Use for example either a) b) or c)
a) The current directory:
imapsync ... --tmpdir .
b) The root directory:
imapsync ... --tmpdir /
c) Any directory you want, like:
imapsync ... --tmpdir "C:\TEMP"
Now, to fix the duplicates on host2, have a run with the option
--delete2duplicates
imapsync ... --delete2duplicates
=======================================================================
Q. On Windows, with --useuid or --usecache a problem occurs with long

121
FAQ.d/FAQ.Various_Software_Servers.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Various_Software_Servers.txt,v 1.13 2019/07/04 09:26:28 gilles Exp gilles $
$Id: FAQ.Various_Software_Servers.txt,v 1.19 2021/01/26 14:05:23 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -18,6 +18,13 @@ Q. From or to Zimbra
R. Read the FAQ.Zimbra.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Zimbra.txt
=======================================================================
Q. From Emailchemy (https://weirdkid.com/emailchemy/)
R. Use option --noabletosearch :
imapsync ... --noabletosearch
=======================================================================
Q. From or to HMailServer version 4.4.1.
@ -51,60 +58,10 @@ Parallells Plesk Panel for Windows requires --sep2 / --prefix2 ""
Parallells Plesk Panel for Linux works with default parameters.
=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail but the tool copies everything in /home/user, how
can i avoid that?
Q. Migrating from or to the Cyrus imap server
Two solutions:
R. Use
imapsync ... --include '^mail'
R. or (better)
imapsync ... --subscribed --subscribe
=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail directory. When imapsync creates the folders in
the new cyrus imap server, it makes a folder "mail" and below that
folder puts all the mail folders the user have in /home/user/mail,
i would like to have all those folders directly under INBOX.
R. Use
imapsync ... --regextrans2 's/^mail/INBOX/' --dry
look at the simulation and if all transformations seem
good then remove the --dry option.
=======================================================================
Q. Migrating from Groupwise to Cyrus
R. By Jamie Neil:
I eventually managed to get the mail to migrate without errors using the
following options:
--maxage 3650
- some messages just don't seem to want to transfer and produce the
perl errors I mentioned before. This prevents the errors, but the
bad messages don't transfer.
Even though the mail migrated OK, there are a couple of gotchas with
Groupwise IMAP:
1) Some of the GW folders are not real folders and are not available
to IMAP, the main problem one being "Sent Items". I could find no way
of coping the contents of these folders. The nearest I got was to
create a "real" folder and copy/move the sent items into it, but
imapsync still didn't see the messages (I think because there is
something funny about the reported dates/sizes).
It think this problem has been rectified in GW6.5.
2) Version 6.0.1 of the Groupwise Internet Connector sucks. I was
getting server aborts when I pushed it a bit hard! I eventually had to
upgrade to 6.0.4 which seems to be a lot more stable.
R. Read the dedicated
https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
=======================================================================
Q. Migrating from iPlanet Messaging Server
@ -131,15 +88,6 @@ You have to extract users logins and passwords in a csv file.
See the "HUGE MIGRATION" section in the README file.
======================================================================
Q: From MailEnable 1.75
R: --sep1 "/" --prefix1 ""
Q: From MailEnable 2.2
R: --sep1 "." --prefix1 ""
Q: To MailEnable
R: --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
======================================================================
Q. From GMX IMAP4 StreamProxy
@ -174,53 +122,6 @@ Q. From or to QQMail IMAP4Server
R. imapsync ... --noabletosearch1
======================================================================
Q. From FirstClass to XXX
http://www.firstclass.com/
R. Migrating from FirstClass is not easy because FirstClass, strangely,
does not show all messages via IMAP. To make it show all messages,
a trick, painful to follow by hand, is moving emails
out and back in, for each folder. May be it can be done by a script.
FirstClass releases prior to release 12 do not shows the "Sent"
folder in IMAP but FirstClass release 12 shows it.
I advice you to upgrade to FirstClass release 12 before leaving it
with imapsync or another imap tool.
Here is a command line used to migrate from FirtClass release 12:
imapsync ... \
--tmpdir /var/tmp --usecache \
--useheader Message-ID \
--idatefromheader \
--addheader \
--regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
--regextrans2 "s/[\^]/_/g" \
--regextrans2 "s/['\"\\\\]/_/g" \
--regextrans2 "s,&AC8-,-,g" \
--regextrans2 "s,&APg-,oe,g"
On Windows:
imapsync.exe ... ^
--automap ^
--usecache ^
--useheader Message-ID ^
--idatefromheader ^
--addheader ^
--regextrans2 "s,(/|^) +,$1,g" ^
--regextrans2 "s, +(/|$),$1,g" ^
--regextrans2 "s/[\^]/_/g" ^
--regextrans2 "s/['\\]/_/g" ^
--regextrans2 "s,^&AC8-,-,g" ^
--regextrans2 "s,^&APg-,oe,g"
Special thanks to Kristian Wind and Joey Alexander for helping me
writing this FAQ item.
See also this worth reading discussion in a Zimbra forum:
http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
======================================================================
Q. From XXX to FTGate

21
FAQ.d/FAQ.Virus.txt
View File

@ -1,5 +1,5 @@
$Id: FAQ.Virus.txt,v 1.6 2019/12/06 09:43:52 gilles Exp gilles $
$Id: FAQ.Virus.txt,v 1.9 2020/01/21 21:35:53 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -17,14 +17,15 @@ Q. My virus scanner claims imapsync.exe is a virus/malware/trojan etc.
R1. Yes, I find the same sometimes with few antivirus, some report that
imapsync might have a trojan in it.
Most virus scanners say imapsync.exe and is ok.
I've done this test on imapsync.exe release 1.727 on the two following
I've done this test on imapsync.exe releases
1.977, 1.945, 1.836 on the two following
meta-virus-scanners:
https://www.virustotal.com/
https://www.metadefender.com/
R2. Explanation:
It may come from the fact that imapsync checks whether there is a
new realease available at http://imapsync.lamiral.info/VERSION
new release available at http://imapsync.lamiral.info/VERSION
It's explained here:
https://imapsync.lamiral.info/#NUMBERS
@ -40,6 +41,20 @@ R4. Using online meta-virus-scanners
https://www.virustotal.com/
https://www.metadefender.com/
Release 1.977
imapsync.exe (64bit) release 1.977 is reported as not clean:
* https://www.virustotal.com/gui/file/8fd16a5eb2aa4d9a037a528654cc0ecdc63936955d289811dee31c6af474f4b0/detection
SecureAge APEX Malicious Jiangmin Backdoor.Kasidet.qt
imapsync_32bit.exe release 1.977 is reported as totally clean:
https://www.virustotal.com/gui/url/94c89ef31190050bebc522f64e32d3f825ef0418cfc424c5847495b9857d57bf/detection
imapsync_bin_Darwin release 1.977 is reported as totally clean:
* https://www.virustotal.com/gui/file/41f107f4f7c86b15c392aef3c69a7f285f3b2a5f02328a1342903eddc5a5a06f/details
* https://metadefender.opswat.com/results#!/file/bzIwMDEwNUJKb2dXa1N3eWVJSDFoZ1d5clBrZ1U/regular/overview
Release 1.945
imapsync.exe release 1.945 is reported as totally clean:

44
FAQ.d/FAQ.XOAUTH2.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.XOAUTH2.txt,v 1.16 2019/07/31 22:58:36 gilles Exp gilles $
$Id: FAQ.XOAUTH2.txt,v 1.17 2021/01/21 13:25:02 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -10,9 +10,47 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
Imapsync tips to use XOAUTH2 authentication (Gmail) and old XOAUTH
=======================================================================
Questions answered in this FAQ are:
Q. Is XOAUTH2 authentication available with imapsync to authenticate
my personnal gmail account? (Quick answer: not yet)
Q. Is XOAUTH2 authentication available with imapsync to authenticate
my personnal Office365 account? (Quick answer: not yet)
Q. Is XOAUTH2 authentication available with imapsync to globally
authenticate gmail users, ie as an admin?
Q. Imapsync XOAUTH2 fails with the following message, how to fix that?
Q. How to use XOAUTH2 via a json file to globally authenticate gmail users?
Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users?
Q. How to use a proxy with XOAUTH2 authentication?
Q. How to use old XOAUTH to globally authenticate gmail users?
Now the questions again with their answers.
=======================================================================
Q. Is XOAUTH2 authentication available with imapsync?
Q. Is XOAUTH2 authentication available with imapsync to authenticate
my personnal gmail account? (Quick answer: not yet)
R. No. But I'm working on that.
XOAUTH2 is a mess or at least, each time I look at it, it leaves
my brain like a mess.
=======================================================================
Q. Is XOAUTH2 authentication available with imapsync to authenticate
my personnal Office365 account? (Quick answer: not yet)
R. No. But I'm working on that too.
=======================================================================
Q. Is XOAUTH2 authentication available with imapsync to globally
authenticate gmail users, ie as an admin?
R. Yes, but XOAUTH2 has been really tested on Unix systems,
less profund on Windows but it should work.
@ -172,7 +210,7 @@ environment variable, if it isn't already set. Example:
=======================================================================
Q. How to use XOAUTH to globally authenticate gmail users?
Q. How to use old XOAUTH to globally authenticate gmail users?
R0. XOAUTH is considered obsolete and superseded by XOAUTH2
Anyway the manage part might be the same (I don't know).

29
FAQ.d/FAQ.Yahoo.txt
View File

@ -1,5 +1,5 @@
$Id: FAQ.Yahoo.txt,v 1.8 2018/05/24 11:34:30 gilles Exp gilles $
$Id: FAQ.Yahoo.txt,v 1.10 2021/04/20 21:47:02 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -14,35 +14,18 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Yahoo.txt
=======================================================================
Q. Synchronizing from Yahoo to XXX
R. Use --host1 imap.mail.yahoo.com --ssl1
R. Use --host1 imap.mail.yahoo.com
./imapsync \
Example:
./imapsync \
--host1 imap.mail.yahoo.com \
--user1 billy \
--password1 secret \
--ssl1 \
--host2 XXX \
--user2 billy \
--password2 secret \
--noabletosearch --errorsmax 10000
SSL is mandatory for yahoo since November 2011.
Why increasing --errorsmax to 10000?
Because during the sync you may encounter several lines like this one:
- msg Inbox/12600 {0} S[36012] F[] I[27-Nov-2015 07:08:35 +0000] could not be fetched: 190 NO [UNAVAILABLE] UID FETCH Server error while fetching messages
- msg Inbox/12600 skipped.
I don't know why it happens, why Yahoo server says
"NO [UNAVAILABLE] UID FETCH Server error while fetching messages"
Usually another run syncs the message successfully.
Why --noabletosearch option?
Yahoo IMAP does not honor well the SEARCH imap command (Dec 2017).
So don't use the --search option.
You can use --minage or/and --maxage but then add also --noabletosearch
Yahoo bandwidth per connection is relatively low, 50 KiB/s to 500 KiB/s max, usually 100 KiB/s.
Consider using --useuid to speed up large accounts.
--noabletosearch
You also need to go to Yahoo, security and enable

66
FAQ.d/FAQ.Zimbra.txt
View File

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Zimbra.txt,v 1.6 2019/06/25 00:15:16 gilles Exp gilles $
$Id: FAQ.Zimbra.txt,v 1.11 2021/04/18 09:15:45 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -15,14 +15,56 @@ https://wiki.zimbra.com/wiki/Guide_to_imapsync
It's obsolete and it will give you, and me, more difficulties than
doing a standard sync without additional options.
Questions answered in this FAQ are:
Q. How to migrate from Zimbra to XXX?
Q. How to migrate from Zimbra with an admin account?
Q. I got this "NO APPEND failed" error:
could not append ( Subject:... Flags:[\Flagged \Seen EmailswithStickyNotes:FLAG51135] )
to folder XXX: 316 NO APPEND failed
What can I do?
Q. Zimbra disconnects without reason
Could not select: * BYE ... Zimbra IMAP4rev1 server closing connection
Now the questions again with their answers.
=======================================================================
Q. From Zimbra to XXX
Q. How to migrate from Zimbra to XXX?
imapsync ... \
--exclude "Conversation Action Settings" \
--exclude "Quick Step Settings" \
--exclude "News Feed"
The basic default (only credendials) should work fine with Zimbra.
=======================================================================
Q. How to migrate from Zimbra with an admin account?
R. Use:
imapsync ... --user1 "normal_user" --authuser1 "admin_user" --password1 "admin_user_password"
To setup or use a Zimbra admin user see:
https://zimbra.github.io/adminguide/8.8.9/index.html#_administrator_accounts
Thanks to Richard Street from thinkround for this tip.
=======================================================================
Q. I got this "NO APPEND failed" error:
could not append ( Subject:... Flags:[\Flagged \Seen EmailswithStickyNotes:FLAG51135] )
to folder XXX: 316 NO APPEND failed
What can I do?
R1.
https://zimbra.github.io/adminguide/8.8.10/index.html
Flagging
Users can create flags and assign them to messages, contacts, and files in Briefcase folders.
In the Admin Console, see COS -> Features -> Mail Features container
Is Flagging on on host2?
If yes, see https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
=======================================================================
Q. Zimbra disconnects without reason
@ -37,17 +79,5 @@ R. The cause of this problem may be that some users have a subscription
Thanks to Danny Herrlitz for this input!
=======================================================================
Q. How to migrate from Zimbra with an admin account?
R. Use:
imapsync ... --user1 "normal_user" --authuser1 "admin_user" --password1 "admin_user_password"
To setup or use a Zimbra admin user see:
https://zimbra.github.io/adminguide/8.8.9/index.html#_administrator_accounts
Thanks to Richard Street from thinkround for this tip.
=======================================================================
=======================================================================

5
FAQ.d/htaccess.txt
View File

@ -1,5 +1,5 @@
# $Id: htaccess.txt,v 1.27 2019/12/08 16:05:27 gilles Exp gilles $
# $Id: htaccess.txt,v 1.29 2020/10/01 10:40:23 gilles Exp gilles $
AddDescription "<b>Back</b> to Imapsync main page." ..
@ -15,6 +15,7 @@ AddDescription "<b>Connection issues</b>." FAQ.Conn
AddDescription "<b>Contacts & Calendars & Chat issues</b>." FAQ.Contacts_Calendars.txt
AddDescription "<b>Dates issues</b>." FAQ.Dates.txt
AddDescription "<b>David Tobit</b> accounts." FAQ.David_Tobit.txt
AddDescription "<b>Docker</b> usage." FAQ.Docker.txt
AddDescription "<b>Domino</b> accounts." FAQ.Domino.txt
AddDescription "<b>Dovecot</b> accounts." FAQ.Dovecot.txt
AddDescription "<b>Duplicated</b> messages issues." FAQ.Duplicates.txt
@ -36,6 +37,7 @@ AddDescription "<b>Migration Plan</b>." FAQ.Migr
AddDescription "<b>The Online UI service Q&R</b>." FAQ.OnlineUI.txt
AddDescription "<b>Password & special characters on Windows</b>." FAQ.Passwords_on_Windows.txt
AddDescription "<b>Password & special characters on Unix</b>." FAQ.Passwords_on_Unix.txt
AddDescription "<b>Password & special characters on Mac OS X</b>." FAQ.Passwords_on_Mac.txt
AddDescription "<b>POP3 issues</b>." FAQ.POP3.txt
AddDescription "<b>Principles & design decisions</b>." FAQ.Principles.txt
AddDescription "<b>Selecting messages</b>." FAQ.Messages_Selection.txt
@ -50,6 +52,7 @@ AddDescription "<b>SSL errors</b>." FAQ.SSL_
AddDescription "<b>SmarterMail</b> accounts." FAQ.SmarterMail.txt
AddDescription "<b>TTL (Time To Live) delay in DNS configuration</b>." FAQ.TTL.txt
AddDescription "<b>Why Imapsync can't do two ways syncs</b>." FAQ.Two_Ways_Sync.txt
AddDescription "<b>Option --usecache and inodes</b>." FAQ.Use_cache.txt
AddDescription "<b>User concurrent access</b>." FAQ.User_Concurrent_Access.txt
AddDescription "<b>Virus scanners on Imapsync</b>." FAQ.Virus.txt