imapsync/FAQ

#!/bin/cat
# $Id: FAQ,v 1.83 2011/01/28 05:14:12 gilles Exp gilles $

+------------------+
| FAQ for imapsync |
+------------------+

=======================================================================
Q. How to install imapsync?

R. http://www.linux-france.org/prj/imapsync/INSTALL

=======================================================================
Q. How to configure imapsync?

R. http://www.linux-france.org/prj/imapsync/README

=======================================================================
Q. Can you give some configuration examples?

R. http://www.linux-france.org/prj/imapsync/FAQ

=======================================================================
Q. How can I have commercial support?

R. Ask the imapsync author and expert: Gilles LAMIRAL
Rates per hour (2010) : 81 euros (111 USD)

=======================================================================
Q. How can I have gratis support?

R. Use the mailing-list

To write on the mailing-list, the address is:
<imapsync@linux-france.org>

To subscribe, send a message to:
<imapsync-subscribe@listes.linux-france.org>

To unsubscribe, send a message to:
<imapsync-unsubscribe@listes.linux-france.org>

To contact the person in charge for the list:
<imapsync-request@listes.linux-france.org>

The list archives may be available at:
http://www.linux-france.org/prj/imapsync_list/
So consider that the list is public, anyone
can see your post. Use a pseudonym or do not
post to this list if you want to stay private.

Thank you for your participation.

=======================================================================
Q. Where I can read up on the various IMAP RFCs?

R. Here:

RFC 3501 - INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1
http://www.faqs.org/rfcs/rfc3501.html

RFC2683 - IMAP4 Implementation Recommendations
http://www.faqs.org/rfcs/rfc2683.html

RFC 2595 - Using TLS with IMAP, POP3 and ACAP
http://www.faqs.org/rfcs/rfc2595.html

RFC 2822 - Internet Message Format
http://www.faqs.org/rfcs/rfc2822.html

RFC 2342 - IMAP4 Namespace
http://www.faqs.org/rfcs/rfc2342.html

RFC2180 - IMAP4 Multi-Accessed Mailbox Practice
http://www.faqs.org/rfcs/rfc2180.html

RFC 4549 - Synchronization Operations for Disconnected IMAP4 Clients
http://www.faqs.org/rfcs/rfc4549.html


=======================================================================
Q. Where I can find old imapsync releases?

R. ftp://www.linux-france.org/pub/prj/imapsync/

=======================================================================
Q. How can I try imapsync with the new Mail::IMAPClient 3.xx perl library?

R. - Download latest Mail::IMAPClient 3.xx at
   http://search.cpan.org/dist/Mail-IMAPClient/
   - untar it anywhere:
        tar xzvf Mail-IMAPClient-3.xx.tar.gz

   - Download latest imapsync at
   http://lamiral.info/~gilles/imapsync/imapsync

   - run imapsync with perl and -I option tailing to use the perl
     module Mail-IMAPClient-3.xx. Example:

         perl -I./Mail-IMAPClient-3.23/lib  ./imapsync ...

     or if imapsync is in directory /path/
        perl -I./Mail-IMAPClient-3.23/lib   /path/imapsync ...

=======================================================================
Q. imapsync does not work with Mail::IMAPClient 3.xx
   How can I downgrade to 2.2.9 release?

R. - Download Mail::IMAPClient 2.2.9 at 
   http://search.cpan.org/~djkernen/Mail-IMAPClient-2.2.9/
   http://search.cpan.org/CPAN/authors/id/D/DJ/DJKERNEN/Mail-IMAPClient-2.2.9.tar.gz
   - untar it anywhere:
        tar xzvf Mail-IMAPClient-2.2.9.tar.gz

   - run imapsync with perl and -I option tailing to use Mail-IMAPClient-2.2.9:
        perl -I./Mail-IMAPClient-2.2.9  ./imapsync [...]

     or if imapsync is in directory /path/
        perl -I./Mail-IMAPClient-2.2.9  /path/imapsync [...]

=======================================================================
Q. Can I use imapsync to migrate emails from pop server to imap server?

R. No. 
You can migrate emails from pop server to imap server with pop2imap:
http://www.linux-france.org/prj/pop2imap/

=======================================================================
Q. I am interested in creating  a local clone of the IMAP on a LAN 
server for  faster synchronisations, email will always be delivered 
to the remote server and so the synchronisation will be one way - from 
remote to local. How suited is imapsync for continuous one-way 
synchronisation of mailboxes? Is there a better solution?

R. If messages are delivered remotely and you play locally with the
copy, in order to have fast access, then the synchronisation can't
be one way. You may change flags, you may move messages in 
different folders etc.

A better tool with this scenario is offlineimap, 
designed for this issue, and faster than imapsync.

=======================================================================
Q. We have found that the sent time and date have been changed to the 
   time at which the file was synchronised. 

R. This is the case with:
   - Eudora
   - Zimbra
   - Outlook 2003
   - Gmail
   but not with 
   - Mutt
   - Thunderbird

Eurora shows by default  the time the imap server received the email. I
think   it is  quite   a  wrong  behavior  since  the messages can have
travelled some time before the reception.

The sent time and date are given by the "Date:" header and it is set
most of the time by the MUA (Mail User Agent, Mutt, Eudora,
Thunderbird etc.).

imapsync does not touch any header since the header is used to
identify the messages in both parts.

Solutions: 
a) use --idatefromheader to set the internal dates on host2 same as the 
   "Date:" headers.
b) In Maildir boxes, after the sync (too late...), use the script
learn/adjust_time.pl to change the internal dates from the "Date:" header.
c) Don't use buggy Eudora.
d) Use the --syncinternaldates option and keep using Eudora.
   --syncinternaldates is now turn on by default.


=======================================================================
Q. imapsync calculates 479 messages in a folder but only transfers 400 
   messages. What's happen?

R. imapsync considers the header part of a message (as a whole or
   only specific lines depending on --useheader --skipheader) 
   to identify a message on both sides.

Two consequences:

  1) Messages with no header are not transferred.
  2) Duplicate messages (identical header) are not transferred
     several times.

The result is that you can have more messages on host1 than on host2.

=======================================================================
Q. Couldn't create [INBOX.Ops/foo/bar]: NO Invalid mailbox name: 
INBOX.Ops/foo/bar 

Example:
sep1=/
sep2=.

imapsync revert each separator automaticaly. 

a) All / character coming from host1 are converted to . (convert the separator)
b) All . character coming from host1 are converted to / (to avoid 
intermediate unwanted folder creation). 

Sometimes the sep1 character is not valid on host2 (character "/" usualy)

R. Try :

  --regextrans2 's,/,X,g'

It'll convert / character to X 
Choose X as you wish: _ or SEP or 
any string (including the empty string).


=======================================================================
Q. The option --subscribe does not seem to work

R. Use it with --subscribed

=======================================================================
Q. Does imapsync retain the \Answered and $Forwarded flags?

R. It depends on the destination server.

a) If the destination server honors the "PERMAENTFLAGS \*" 
directive or no PERMAENTFLAGS at all then imapsync synchronises 
all flags except the flag \Recent
(RFC 3501 says "This flag can not be altered by the client.").

b) If the destination server honors the "PERMAENTFLAGS without the 
special "\*" (meaning it accepts any flag) then imapsync synchronises 
only the flags listed in PERMANENTFLAGS.

Some imap servers have problems with flags not beginning with
the backslash character \ 
(see next question to find a solution to this issue)


=======================================================================
Q. I need to keep only a defind list of flags, how can I do?
The destination imap server complains about bad flags (Exchange).

R. For example if you want to keep only the following flags
\Seen \Answered \Flagged \Deleted \Draft
then use these magic --regexflag options (thanks to Phil):

 --regexflag 's/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg'

Analysis is left to the reader.

This one is longer and may be use with old perl (no /e regex extension):
 --regexflag 's/(.*)/$1 jrdH8u/' \
 --regexflag 's/.*?(\\Seen|\\Answered|\\Flagged|\\Deleted|\\Draft|jrdH8u)/$1 /g' \
 --regexflag 's/(\\Seen|\\Answered|\\Flagged|\\Deleted|\\Draft|jrdH8u) (?!(\\Seen|\\Answered|\\Flagged|\\Deleted|\\Draft|jrdH8u)).*/$1 /g' \
 --regexflag 's/jrdH8u *//'


======================================================================
Q. imapsync fails with the following error:
flags from : [\Seen NonJunk]["10-Aug-2006 13:00:30 -0400"]
Error trying to append string: 58 NO APPEND Invalid flag list

R. For some servers, flags have to begin with a \ character.
The flag "NonJunk" may be a invalid flag for your server 
so use for example:

imapsync ...    --regexflag 's/NonJunk//g'

Remark (thanks to Arnt Gulbrandsen): 
IMAP system flags have to begin with \ character.
Any other flag must begin with another character.
System flags are just flags defined by an RFC instead of by users.
Conclusion, some imap server coders don't read the RFCs (so do I).

recent imapsync deals with this issue by filter with PERMANENTFLAGS
automatically.

=======================================================================
Q. Flags are not well synchonized. Is it a bug?

R. It happens with some servers on the first sync. 
Also, it was a bug from revision 1.200 to revision 1.207

Solution: run imapsync a second time. imapsync synchronizes flags
on each run unless option --fast is used.

=======================================================================
Q. imapsync hangs taking up 99.8% cpu right after start, 
   after printing imapd doesn't support MD5 auth.

R. Try option --noauthmd5

=======================================================================
Q. Some passwords contain * and " characters. Login fails.
R. Use a backslash to escape the characters:

    imapsync --password1 \"password\" 

It works for the star * character,
I don't know if it works for the " character.

=======================================================================
Q. Out of memory on FreeBSD

R. http://groups.google.com/group/lucky.freebsd.questions/browse_thread/thread/f4218e4252863328

See the user limit with the command
    ulimit -a
To change it, try 
    ulimit -d 1000000000  
Also
http://www.unixadmintalk.com/f41/perl-out-memory-sbrk-9112/
The default hard datasize limit on FreeBSD is 512MB. To raise it, put this 
(or more) in /boot/loader.conf and reboot:

kern.maxdsiz="1024M"

=======================================================================
Q. With huge account (many messages) when it comes to reading the
destination server it comes out this error:
"To   Folder [INBOX.foobar] Not connected" 
What can I do?

R. May be spending too much time on the source server, the connection 
timed out on the destination server.
Try options :
  --nofoldersizes --useheader Message-ID --fast

recent imapsync reconnects automatically.

=======================================================================
Q. imapsync failed with a "word too long" error from the imap server,
   What can I do?

R. Use imapsync release 1.172 or at least 1.166 with options
   --split1 500 --split2 500
   or a old old imapsync (before 1.94)

=======================================================================
Q. Does imapsync support IMAP over TLS (IMAPS)?

R. Yes natively since release 1.161. 
still, 2 ways, at least :

a) Use --ssl1 and/or --ssl2 options

--ssl1 tells imapsync to use ssl on host1
--ssl2 tells imapsync to use ssl on host2


b) Use stunnel
   http://www.stunnel.org/
   Assuming there is an imaps (993) server on imap.foo.org,
   on your localhost machine (or bar machine) run :
       stunnel -c -d imap -r imap.foo.org:imaps
   or using names instead of numbers 
       stunnel -c -d 143 -r imap.foo.org:993
   then use imapsync on localhost (or bar machine) imap (143) port.


=======================================================================
Q: How to have an imaps server?

R. 
a) Install one 

b) or use stunnel :
   Assuming there is an imap (143) server on localhost
        stunnel  -d 993 -r 143 -f

c) or use stunnel on inetd
   imaps stream  tcp nowait cyrus /usr/sbin/stunnel -s cyrus -p /etc/ssl/certs/imapd.pem -r localhost:imap2

=======================================================================
Q: I'm trying to use imapsync on win32 for gmail, but it requires ssl,
or at least claims to. Imapsync appears to require io-socket-ssl, 
which doesn't seem to be available on win32. Are there any other
options?

R: (Q and R come as is from Bryce Walter)

I think I'm having success using cygwin perl instead of ActiveState
Perl. I wasn't able to get CPAN working and building IO::Socket::SSL
in ActiveState, but cygwin did all right. I had to force the install
of the Net::SSLeay dependency, because it partially failed one test,
but I think it worked anyway. In order to get working in cygwin, I
installed the entire "perl" category, lynx, ncftp, and lftp (specified
as ftp program in cpan setup). I'm not sure if I needed all those, or
if cpan just kept asking because I didn't have any installed at the
time. Anyway, cpan worked, and I installed all dependencies that
imapsync complained about until it started working. 

=======================================================================
Q: Multiple copies when I run imapsync twice ore more.

R. Multiple copies of the emails on the destination server. Some IMAP
servers (Domino for example) add some headers for each message
transfered. The message is transfered again and again each time you
run imapsync. This is bad of course. The explanation is that imapsync
considers the message is not the same since headers have changed (one
line added) and size too (the header part). 

You can look at the headers found by imapsync by using the --debug
option (and search for the message on both part), Header lines from
the source server begin with a "FH:" prefix, Header lines from the
destination server begin with a "TH:" prefix. Since --debug is very
verbose I suggest to isolate a email in a specific folder in case you
want to forward me the output.

The way to avoid this problem is by using options --skipheader and
--skipsize, like this (avoid headers beginning whith the string "X-"):

 imapsync ... --skipheader '^X-' --skipsize

To skip several headers you can use --skipheader one time

 imapsync ... --skipheader '^X-|^Status|^Bcc'

If you think you have too many header to avoid just use

 imapsync ... --useheader 'Message-ID' --skipsize

Remark. (Trick found by Tomasz Kaczmarski)

Option --useheader 'Message-ID' asks the server to send only header
lines begining with 'Message-ID'.  Some (buggy) servers send the whole
header (all lines) instead of the 'Message-ID' line. In that case, a
trick to keep the --useheader filtering behavior is to use
--skipheader with a negative lookahead pattern :

 imapsync ... --skipheader '^(?!Message-ID)' --skipsize

 Read it as "skip every header except Message-ID".

======================================================================
Q. I am transferring mails from one IMAP server to another. I am using
   an SSL connection. Transferring huge mails (>10MB) takes ages.

R. try to transfer the mails without SSL connection. SSL code outside
   imapsync uses a memory buffer, which gets increased upon reading of
   mails by 4096 bytes. This creates a huge load on the host imapsync
   runs on by copying the memory buffers for every 4096 byte step.
   This does not occur without SSL.

(Written by Stefan Schmidt)

======================================================================
Q. What are --subscribe and --subscribed for, and how can they be used?

R. In the IMAP protocol each user can subscribe to one or more folders.
   Then he can configure its email software to just see his subscribed
   folders list. That's an IMAP feature.

   Knowing that, the imapsync help says:

  imapsync --help
  ...
  --subscribed           : transfers subscribed folders.
  --subscribe            : subscribe to the folders transferred on the 
                           host2 that are subscribed on host1.
  --subscribe_all        : subscribe to the folders transferred on the 
                           host2 even if they are not subscribed on host1.


======================================================================
Q. I want to exclude a folder hierarchy like "public"

R. Use:

--exclude '^public\.'
or maybe
--exclude '^"public\.'

In the example given the character "." is the folder separator, you
can ommit it. Just take the string as it appears on the imapsync
output line :

From folders list : [INBOX] [public.dreams] [etc.]

======================================================================
Q. I want to exclude only INBOX

R. Use:

  imapsync ... --exclude '^INBOX$' 

A good way to see what will be done is to first use:

  imapsync ...  --exclude '^INBOX$' --justfolders --nofoldersizes --dry


======================================================================
Q. I want the --folder 'MyFolder' option be recursive.

Two solutions:

R. Use

--folderrec 'MyFolder'

R. Do not use the --folder option. 
   Instead, use --include '^MyFolder'
   Then the folder "MyFolder" and all its subfolders will be handled
   and only them.


======================================================================
Q. How to migrate from or to Exchange 2007/2010 with an 
   admin/authuser account?
   
R. The trick comes from Michele Marcionelli:

This doesn't work:
imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ...

This works:
imapsync ... --user2 'domain\admin2\user2' --password2 adminpassword2 ...


======================================================================
Q. How to migrate from uw-imap with an admin/authuser account?

R. Use 

--user1="user*admin_user" --password1 "admin_user_password"


======================================================================
Q. How to migrate from cyrus with an admin account?

R. Use 
--authuser1 admin_user  ----password1 admin_user_password \
 --user1 foo_user --ssl1

In this case, --authmech1 PLAIN will be used by default since it
is the only way to go for now. So don't use --authmech1 SOMETHING
with --authuser1 admin_user, it will not work.
Same behavior with the --authuser2 option.

Do not forget the option --ssl1 since PLAIN auth is only
supported with ssl encryption most of the time. But it can
work without --ssl1 if PLAIN is permitted in normal use.

Here is an example:
 imapsync \
	--host1 server1 \
	--user1 joe \
	--authuser1 AdminAccount \
	--password1 AdminAccountPassword \
	--ssl1 \
	--host2 server2 \
	--user2 joe \
	--password2 joespassonserver2 \
	--exclude '^user\.'

======================================================================
Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
Netscape servers with an admin account?

R: Those imap servers don't allow the typical use of --authuser1 to use an
administrative account. They expect the use of an IMAP command called
proxyauth that is issued after login in as an administrative account.

For example, consider the administrative account 'administrator' and your
real user 'real_user'. The IMAP sequence would be:

  OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS
  CHILDREN BINARY UNSELECT LANGUAGE STARTTLS XSENDER X-NETSCAPE XSERVERINFO
  AUTH=PLAIN] imap.server IMAP4 service (Sun Java(tm) System Messaging
  Server ...))
  1 LOGIN administrator password
  1 OK User logged in
  2 PROXYAUTH real_user
  2 OK Completed

In imapsync, you can achieve this by using the following options:

  --host1 source.imap.server \
  --user1 real_user \
  --authuser1 administrator \
  --proxyauth1 \
  --passfile admin.txt

======================================================================
Q. Is there anyway of making imapsync purge the destination folder
   when the source folder is deleted?

R. No, that's too dangerous. May be coded in future release.

But if the source folder is empty (not deleted) and options --delete2
--expunge2 are used then the destination folder will be empty.

======================================================================
Q. Is it possible to synchronize all messages from one server to 
another without recreating the folder structure and the target server.

R. Yes.

For example, to synchronize all messages in all forders on host1
to folder INBOX only on host2:

1) First try (safe mode):

imapsync \
 ...
 --regextrans2 's/(.*)/INBOX/' \
 --dry --justfolders

2) See if the output says everything you want imapsync to do, 
   --dry option is safe and does nothing real.
   
3) Remove --dry 
   Check the imap folder tree on the target side, you should
   only have one: the classical INBOX.

4) Remove --justfolders


======================================================================
Q. I have moved from Braunschweig to Graz, so I would like to have my
   whole Braunschweig mail sorted into a subfolder INBOX.Braunschweig 
   of my new mail account.

R.
1) First try (safe mode):

imapsync \
 ...
 --regextrans2 's/INBOX(.*)/INBOX.Braunschweig$1/' \
 --dry --justfolders

2) See if the output says everything you want imapsync to do, 
   --dry option is safe and does nothing real.
   
3) Remove --dry 
   Check the imap folder tree on the target side

4) Remove --justfolders

=======================================================================
Q. Give examples about --regextrans2

R. --regextrans2 is used to transform folder names

Examples: 

0) First try with --dry --justfolders options since imapsync shows the
   transformations it will do without really doing them. Then when
   happy with the output remove the --dry --justfolders options.

1) To remove INBOX. in the name of destination folders:

    --regextrans2 's/^INBOX\.(.+)/$1/'

2) To sync a complete account in a subfolder called FOO:
  
  a) Seperator is dot character "." and "INBOX" prefixes every folder
  
    --regextrans2 's/^INBOX(.*)/INBOX.FOO$1/'

  or:
  
  b) Seperator is slash character "/" and there is no prefix 
  
   --regextrans2 's#(.*)#FOO/$1#'
   
   or:
   
   
  c) Any separator, any prefix solution, FOO is the subfolder:

  It is a complicated line because every case is taken into account.
  Type it in one line (or with the \ at the end of first line on Unix shells.
   
   --regextrans2 's,${h1_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \
   --regextrans2 's,^INBOX$,${h2_prefix}NEW${h2_sep}INBOX,'


3) to substitute all characters dot "." by underscores "_"
    --regextrans2 's/\./_/g'

4) to change folder names like this:
[mail/Sent Items] -> [Sent]
[mail/Test]       -> [INBOX/Test]
[mail/Test2]      -> [INBOX/Test2]

    --regextrans2 's#^mail/Sent Items$#Sent#' \
    --regextrans2 's#^mail/#INBOX/#'

=======================================================================
Q. I would like to move emails from InBox to a sub-folder called,
   say "2005-InBox" based on the date (Like all emails received in the
   Year 2005 should be moved to the folder called "2005-InBox").

R. 2 ways :

a) Manually:
------------

1) You create a folder INBOX.2005-INBOX

2) Mostly every email software allow sorting by date. In inbox, you
   select from 1 january to 31 december messages with the shift key.
   (in mutt, use ~d)

3) Cut/paste in INBOX.2005-INBOX

b) With imapsync:
-----------------

You have to calculate the day of year (and add 365).  For example,
running it today, Sat Mar 11 13:06:01 CET 2006:

imapsync ... 
--host1 imap.truc.org --host2 imap.trac.org \
--user1 foo --user2 foo  \
...
--maxage 435 --minage 70 \
--regextrans2 's/^INBOX$/INBOX.2005-INBOX/' \
--folder INBOX


To know the day of year:

$ date
Sat Mar 11 13:06:01 CET 2006

$ date +%j
070

Also, you must take imapsync 1.159 at least since I tested what I just
wrote above and found 2 bugs about --mindate --maxdate options
behavior.

=======================================================================
Q. I want to play with headers line and --regexmess but I want to leave
   the body as is

R. The header/body separation is a blank line so an example:
   --regexmess 's{\A(.*?(?! ^$))^Date:(.*?)$}{$1Date:$2\nX-Date:$2}gxms'

Will replace (HeaderBegin and HeaderEnd are not part of the header)

HeaderBegin
Message-ID: <499EF800.4030002@blabla.fr>
Date: Fri, 20 Feb 2009 19:35:44 +0100
From: Gilles LAMIRAL <lamiral@linux-france.org>
HeaderEnd

by

HeaderBegin
Message-ID: <499EF800.4030002@blabla.fr>
Date: Fri, 20 Feb 2009 19:35:44 +0100
X-Date: Fri, 20 Feb 2009 19:35:44 +0100
From: Gilles LAMIRAL <lamiral@linux-france.org>
HeaderEnd

This example just add an header line "X-Date:" based on "Date:" line.

=======================================================================
Q. My imap server does not accept a message and warns
   "Invalid header". What is the problem?

R. You fall in the classical mbox versus Maildir/ format
   problem. May be you use a misconfigured procmail rule.

A header beginning like the following one is in the mbox
format, header line 1 has no colon behind "From", header
lines 2 through N do have a colon :

From foo@yoyo.org  Sat Jun 22 01:10:21 2002
Return-Path: <foo@yoyo.org>
Received: ...

Any Maildir/ configured imap server may refuse this message since its
header is invalid. The first "From " line is not valid. It lacks a
colon character ":". To solve this issue you have several solutions

a) Remove manually this first "From " line for each message before
   using imapsync. 

b) Replace manually the whitespace by a colon in string "From " but you
   might end with two "From:" lines (just have a look at the other 
   header lines of the message)

c) Run imapsync with the following option (this replaces "From "by "From:"):
   --regexmess 's/\AFrom /From:/'

or may be better (no other "From:" collision):

d) Run imapsync with the following option (this replaces "From "by "X-om:"):
   --regexmess 's/\AFrom /X-om:/'

e) Run imapsync with the following option (this removes the whole "From " line):
   --regexmess 's{\AFrom\ [^\n]*(\n)?}{}gxms'

Solution e) is solution a) made by imapsync itself.
Solutions c) and d) keep "From " lines information
(normally it's useless to keep them)

Best solutions are e) or d).

=======================================================================
Q. The contact folder isn't well copied. 
   How to copy the contact folder?

R. Forget the destination server (choose the same)
Change the script around line 1426
       # ITSD
       $new_id = $from->copy($t_fold,$f_msg);
       #$new_id = $to->append_string($t_fold,$string, $flags_f, $d);

and tried a copy of the mail instead an append_string. Because we are
using the same server, we can use $from->copy Therefore we seem to not
download and upload the message and therefore we do not have any
format issues. And now it works fine. (Thanks to Hansjoerg.Maurer)

=======================================================================
              Server specific issues and solutions
=======================================================================

Q. From or to HMailServer version 4.4.1.

R. You have to add prefix and separator manually because  4.4.1 doesn't
honor the NAMESPACE imap command.

Example for host1:

imapsync ... \
         --prefix1 "" --sep1 .

No specific option for HMailServer 5.3.3 since NAMESPACE is supported.

Maybe --subscribe_all will help you to see all migrated folders.


=======================================================================
Q. Synchronising from SmarterMail to XXX

imapsync --host1 imap.d1.org --user1 joe --password1 secret1 --sep1 "/" \
         --host2 imap.d2.org --user2 joe --password2 secret2  \
         --noauthmd5 \
         --prefix1 "Inbox/"  \
         --regextrans2 's#^Inbox$#INBOX#' \
         --regextrans2 's#Sent Items$#Sent#' \
         --dry --justfolders

Maybe add other --regextrans2 to change folder names and see the result.
When satisfied, run without --dry --justfolders

=======================================================================
Q. Synchronising from XXX to Gmail

R. There are some details to get the special [Gmail] sub-folders
   right. Here's an example of migrating an old "Sent" folder to
   Gmail's structure:

imapsync --host1 mail.oldhost.com \
         --user1 my_email@oldhost.com \
         --password1 password \
         --host2 imap.gmail.com --ssl2 \
         --user2 my_email@gmail.com \
         --password2 password \
         --useheader 'Message-Id' \
         --prefix2 '[Gmail]/' \
         --folder 'INBOX.Sent' \
         --regextrans2 's/Sent/Sent Mail/'

The same goes for the "All Mail" archive pseudo-folder.

=======================================================================
Q. Synchronising from Gmail to XXX

R. Gmail needs SSL 

./imapsync \
           --host1 imap.gmail.com \
           --ssl1 \
           --authmech1 LOGIN \
           --user1 gilles.lamiral@gmail.com \
           --password1 gmailsecret \
           --host2 localhost
           --user2 tata \
           --password2 tatasecret \
           --useheader="X-Gmail-Received" \
           --useheader 'Message-Id'

If your destination imap server doesn't like "[Gmail]" name, just add
option:
           --regextrans2 's/\[Gmail\]/Gmail/'


=======================================================================
Q. migrate email from gmail to google apps

R. Take a look at:
http://www.linux-france.org/prj/imapsync_list/msg00639.html

http://biasecurities.com/blog/2009/migrate-email-from-gmail-to-google-apps/
http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration/

=======================================================================
Q. from Microsoft's Exchange 2007 to Google Apps for your Domain 
   (GAFYD)

R. Take a look at:
http://mark.ossdl.de/2009/02/migrating-from-exchange-2007-to-google-apps-mail/

=======================================================================
Q. Syncing from Google Apps domain to Googlemail account

A known bug encountered with this output (Alexander is a folder name):

++++ Verifying [Alexander] -> [Alexander] ++++
+ NO msg #16 [A96Dh4AwlLVphOAW5MS/eQ:779824] in Alexander
+ Copying msg #16:779824 to folder Alexander
flags from : [\Seen]["04-Jul-2007 14:32:22 +0100"]
Couldn't append msg #16 (Subject:[Rieter-Event    (please accept with
comments)]) to folder Alexander: 46 NO Invalid folder: Sent (Failure)

In fact folder "Sent" is just the last folder listed previously
as a:
...
To   Folder [Sent]                             does not exist yet
To   Folder [Sonja]                             Size:   1024546 Messages:    96
...

R. Just run imapsync a time like this :
imapsync ... --folder Alexander

=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
   /home/user/mail but the tool copies everything in /home/user, how
   can i avoid that?

Two solutions:

R. Use
   imapsync ... --include '^mail'

R. or (better)
   imapsync ... --subscribed --subscribe


=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
   /home/user/mail directory.  When imapsync creates the folders in
   the new cyrus imap server, it makes a folder "mail" and below that
   folder puts all the mail folders the user have in /home/user/mail,
   i would like to have all those folders directly under INBOX.

R. Use 
   imapsync ... --regextrans2 's/^mail/INBOX/' --dry
   look at the simulation and if all transformations seem
   good then remove the --dry option.


=======================================================================
Q. Migrating from Groupwise to Cyrus

R. By Jamie Neil:

I eventually managed to get the mail to migrate without errors using the 
following options:

--sep1 /
  - doesn't report separator so has to be set explicitly.

--nosyncacls
  - doesn't support ACLs.

--skipheader '^Content-Type'
  - MIME separator IDs seem to change every time a mail is accessed so 
    this is required to stop duplicates.

--maxage 3650
  - some messages just don't seem to want to transfer and produce the 
    perl errors I mentioned before. This prevents the errors, but the
    bad messages don't transfer.

Even though the mail migrated OK, there are a couple of gotchas with 
Groupwise IMAP:

1) Some of the GW folders are not real folders and are not available
to IMAP, the main problem one being "Sent Items". I could find no way
of coping the contents of these folders. The nearest I got was to
create a "real" folder and copy/move the sent items into it, but
imapsync still didn't see the messages (I think because there is
something funny about the reported dates/sizes).

It think this problem has been rectified in GW6.5.

2) The "skipheader '^Content-Type'" directive is required to stop 
duplicate messages being created. GW seems to generate this field on
the fly for messages that have MIME separators and so it's different
every time.

3) Version 6.0.1 of the Groupwise Internet Connector sucks. I was 
getting server aborts when I pushed it a bit hard! I eventually had to
upgrade to 6.0.4 which seems to be a lot more stable.


=======================================================================
Q. Migrating from iPlanet Messaging Server
5.2 Patch 2 (built Jul 14 2004)) to Groupwise 7.0
I encounter many errors like this:
"Error trying to append string: 17847 BAD APPEND"

R. GroupWise 7 seems buggy. Apply GroupWise 7 support pack 1


=======================================================================
Q. Migrating from David Tobit V8

R. Use the following options :
imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed 

=======================================================================
Q. Migrating from Tobit David Server 6 
   ("DvISE Mail Access Server MA-6.60a (0118)")

R. Look at the discussion:
http://www.linux-france.org/prj/imapsync_list/msg00582.html
http://www.linux-france.org/prj/imapsync_list/threads.html#00582
patch saved in ./patches/imapsync-1.337_tobit_V6.patch


=======================================================================
Q. I need to migrate 1250 mailboxes from one cyrus-IMAP server to another
(empty) one. (Box-swap). The passwords are in a MySQL Database.
Can you tell me if your script suits my needs?

R. Partially, mailboxes must exist before running imapsync.
May be, Box-swap is not listed in "Failure stories"
nor "Success stories" in the README file.
You have to extract user and password in a csv file.
See the "HUGE MIGRATION" section in the README file.

======================================================================
Q. From Cyrus to Notes

Juhu! --useheader 'Message-ID' --skipsize does wonders! :)

======================================================================
Q. From cyrus to dbmail

R. (Given by Michael Monnerie, left as is)

dbmail creates a header like
X-DBMail-PhysMessage-ID:94348
on messages. There was another problem with cyrus, other headers get 
modified also, so I recommend this for a full sync:

imapsync --host1 cyrushost  --user1 u1 --password1 p1 \
         --host2 dbmailhost --user2 u2 --password2 p2 \
         --useheader 'Message-ID' \
         --skipsize \
         --syncinternaldates

I needed it because there are obviously some 
encoding differences between the two servers, and modified headers lead 
to all messages being doubled. Examples:

dbmail:
Content-Type:text/plain; format=flowed; charset=iso-8859-1; 
reply-type=original
cyrus:
Content-Type:text/plain; format=flowed; charset="iso-8859-1"; 
reply-type=original

And also the "Received" headers got blanks added in DBmail, etc.

******************

For a full server copy from cyrus to dbmail I used:
imapsync --host1 cyrus  --user1 x --authuser1 x --password1 x --ssl1  \
         --host2 dbmail --user2 x --authuser2 x --password2 x \
         --skipheader '(^X-|^Received|^MIME-|^Content-Type|^Disposition-|^From|^Cc|^Reply-|^Subject|^To|^DomainKey).*' \
         --skipsize \
         --sep1 '/' --exclude 'user/demo/Trash' \
         --regextrans2 's/^user.//' --syncinternaldates

The 'exclude user/demo/Trash' was used because there was one message 
there with 8 bit headers which dbmail doesn't accept, so I had to skip
the whole folder. It would be nice to have an option to just ignore
and log unsyncable messages, but do the rest, instead of stopping.

******************

There are two other major problems:

1) dbmail doesn't accept utf8 header, while cyrus does. imapsync stops
in that case, making sync impossible

To convert the whole messages from 8bit to 7bit, use option :

--regexmess 's/[\x80-\xff]/X/g'

(This is bad since only headers need this).

======================================================================
Q: From MailEnable 1.75
R: --sep1 "/" --prefix1 ""

Q: From MailEnable 2.2
R: --sep1 "." --prefix1 ""

======================================================================
Q. From GMX IMAP4 StreamProxy
R. Use:
   --prefix1 INBOX and --sep1 .

======================================================================
Q. From Courier to Archiveopteryx
R. http://www.archiveopteryx.org/migration/imapsync
Use:
   --useheader Message-Id --skipsize

======================================================================
Q. To Sun Java(tm) System Messaging Server 6.2-7.05
Q. To Communigate Pro - Solaris version

R. See and run patches/imapsync_1.267_jari

======================================================================
Q. From any to Exchange2007 

Several problems:
- Big messages: increase the "send- and receive-connector" 
  in exchange2007 to 40 MB.

R. 2 solutions

R1. With imapsync (only partial success)
--skipsize --skipheader 'Received' \
--regexmess 's{\A(.*?(?!^$))^Date:(.*?)$}{$1Date:$2\nReceived: From; $2}gxms'
Any user having time to spend to debug Exchange2007 with imapsync
is welcome.

R2. Other solution
Two users succeded by using "MS Transporter Suite" (which is closed
expensive nonfree software).

======================================================================
Q. From Microsoft Exchange 2000 IMAP4rev1 server version 6.0.6487.0.

R. imapsync ... \
   --prefix1 INBOX. --prefix2 INBOX. --syncinternaldates --subscribe \
   --maxsize 10485760

======================================================================
Q: How can I write an .rpm with imapsync
R: I don't know but Neil Brown wrote one rpm package and you'll find
   his .spec file here :
   http://www.linux-france.org/prj/imapsync/learn/rpm/

======================================================================
Q: Problems on win32, Timezone and Date::Manip
R: Comment the code like the following

  if ($syncinternaldates) {
     $d = $f_idate;
     $debug and print "internal date from 1: [$d]\n";
     #require Date::Manip;
     #Date::Manip->import(qw(ParseDate Date_Cmp UnixDate));
     #$d = UnixDate(ParseDate($d), "%d %b %Y %H:%M:%S %z");
     #$d = "\"$d\"";
     #$debug and print "internal date from 1: [$d] (fixed)\n";
  }