#!/bin/cat # $Id: FAQ,v 1.102 2012/01/08 06:40:50 gilles Exp gilles $ +------------------+ | FAQ for imapsync | +------------------+ Unix versus Windows syntax. A) \ versus ^ On Unix shells you can write a single command on multiple lines by using the escape character \ at the end of each line (except the last one). On Windows this character is ^ Unix example: ./imapsync \ --host1 imap.truc.org --user1 foo --password1 secret1 \ --host2 imap.trac.org --user2 bar --password2 secret2 Windows example: imapsync ^ --host1 imap.truc.org --user1 foo --password1 secret1 ^ --host2 imap.trac.org --user2 bar --password2 secret2 Of course you can write the command on one only line without characters \ nor ^. I use them because the output is better, no truncation, pretty print. It's just sugar. In this FAQ I use \ for examples. Transcript to ^ if you're on a Windows system. B) ' versus " On Windows the single quote character ' doesn't work like on Unix so in the examples of this FAQ the command containing single quotes ' will fail on Windows. To fix it just replace single quotes ' by double quotes " ======================================================================= Q. How to install imapsync? R. Read the INSTALL file in the tarball also available at http://imapsync.lamiral.info/INSTALL ======================================================================= Q. How to configure and run imapsync? R. Read the README and FAQ files in the tarball also available at http://imapsync.lamiral.info/README http://imapsync.lamiral.info/FAQ ======================================================================= Q. Can you give some configuration examples? R. The FAQ file contains many examples for several scenarios http://www.linux-france.org/prj/imapsync/FAQ ======================================================================= Q. How can I have commercial support? R. Buy support from imapsync author and expert: Gilles LAMIRAL http://imapsync.lamiral.info/#buy_support ======================================================================= Q. How can I have gratis support? R. Use the mailing-list To write on the mailing-list, the address is: To subscribe, send a message to: To unsubscribe, send a message to: To contact the person in charge for the list: The list archives may be available at: http://www.linux-france.org/prj/imapsync_list/ So consider that the list is public, anyone can see your post. Use a pseudonym or do not post to this list if you want to stay private. Thank you for your participation. ======================================================================= Q. I need to migrate hundred accounts, how can I do? R. If you have many mailboxes to migrate think about a little shell program. Write a file called file.txt (for example) containing users and passwords. The separator used in this example is ';' The file.txt file contains: user001_1;password001_1;user001_2;password001_2 user002_1;password002_1;user002_2;password002_2 user003_1;password003_1;user003_2;password003_2 user004_1;password004_1;user004_2;password004_2 user005_1;password005_1;user005_2;password005_2 ... On Unix the shell program can be: { while IFS=';' read u1 p1 u2 p2; do imapsync --host1 imap.side1.org --user1 "$u1" --password1 "$p1" \ --host2 imap.side2.org --user2 "$u2" --password2 "$p2" ... done ; } < file.txt On Windows the batch program can be: FOR /F "tokens=1,2,3,4 delims=; eol=#" %%G IN (file.txt) DO imapsync ^ --host1 imap.side1.org --user1 %%G --password1 %%H ^ --host2 imap.side2.org --user2 %%I --password2 %%J ... The ... can be replaced by nothing or any supplementary imapsync option. ======================================================================= Q. Where I can find old imapsync releases? R. Search the internet. ======================================================================= Q. Where I can find free open and gratis imapsync releases? R. Search the internet. Q. Is is legal? R. Yes, the license permits it http://imapsync.lamiral.info/COPYING ======================================================================= Q. I use --useuid which uses a cache in /tmp or --tmpdir, the hostnames host1 or host2 has changed but mailboxes are the same. Will imapsync generate duplicate messages on next runs? R. Yes Q. How can I fix this? R. The cache path reflects hostnames or ip adresses, just change the directory names of host1 or host2. Use --dry to see if next runs will generate duplucates. By default the cache is like /tmp/imapsync_cache/host1/user1/host2/user2/... ======================================================================= Q. How can I speed up transfers? R. By using --useuid imapsync avoid getting messages headers and build a cache. On Unix a good thing is to add also --tmpdir /var/tmp to keep the cache since /tmp is often cleared on reboot. imapsync ... --useuid R. Add also --nofoldersizes since the default behavior is to compute folder sizes. Folder sizes are useless for the transfer, just useful to see what has to be done on each folder. ======================================================================= Q. I see warning messages like "Host1 Sent/15 size 1428 ignored (no header so we ignore this message)" What can I do to transfer those messages? R1. Use --addheader option, it will add a header like "Message-Id: <15@imapsync>" and transfer the message on host2. Duplicates won't happen in next runs. imapsync ... --addheader R2. Use --useuid then imapsync will avoid dealing with headers. imapsync ... --useuid ======================================================================= Q. How can I try imapsync with the new Mail::IMAPClient 3.xx perl library? R. - Download latest Mail::IMAPClient 3.xx at http://search.cpan.org/dist/Mail-IMAPClient/ - untar it anywhere: tar xzvf Mail-IMAPClient-3.xx.tar.gz - Get any imapsync (latest is better). - run imapsync with perl and -I option tailing to use the perl module Mail-IMAPClient-3.xx. Example: perl -I./Mail-IMAPClient-3.30/lib ./imapsync ... or if imapsync is in directory /path/ perl -I./Mail-IMAPClient-3.30/lib /path/imapsync ... - Look at the script named i3 in the tarball, it can be used to run imapsync with included Mail-IMAPClient-3.30/ wherever you unpacked the imapsync tarball ======================================================================= Q. How can I use imapsync with Mail::IMAPClient 2.2.9 perl module? R. - Download Mail::IMAPClient 2.2.9 at http://search.cpan.org/~djkernen/Mail-IMAPClient-2.2.9/ http://search.cpan.org/CPAN/authors/id/D/DJ/DJKERNEN/Mail-IMAPClient-2.2.9.tar.gz - untar it anywhere: tar xzvf Mail-IMAPClient-2.2.9.tar.gz - run imapsync with perl and -I option tailing to use Mail-IMAPClient-2.2.9: perl -I./Mail-IMAPClient-2.2.9 ./imapsync [...] or if imapsync is in directory /path/ perl -I./Mail-IMAPClient-2.2.9 /path/imapsync [...] - Look at the script named i2 in the tarball, it can be used to run imapsync with included Mail-IMAPClient-2.2.9/ wherever you unpacked the imapsync tarball ======================================================================= Q. Can I use imapsync to migrate emails from pop3 server to imap server? R1. No. You can migrate emails from pop server to imap server with pop2imap: http://www.linux-france.org/prj/pop2imap/ R2. Yes Many pop3 servers runs in parallel with an imap server on the exactly the same mailboxes. They serve the same INBOX (imap serves INBOX and several other folders, pop3 serves only INBOX) So have a try with imapsync on the same host1. ======================================================================= Q. I am interested in creating a local clone of the IMAP on a LAN server for faster synchronisations, email will always be delivered to the remote server and so the synchronisation will be one way - from remote to local. How suited is imapsync for continuous one-way synchronisation of mailboxes? Is there a better solution? R. If messages are delivered remotely and you play locally with the copy, in order to have fast access, then the synchronisation can't be one way. You may change flags, you may move messages in different folders etc. A better tool with this scenario is offlineimap, designed for this issue, and faster than imapsync. ======================================================================= Q. We have found that the sent time and date have been changed to the time at which the file was synchronised. R. This is the case with: - Eudora - Zimbra - Outlook 2003 - Gmail but not with - Mutt - Thunderbird Eurora shows by default the time the imap server received the email. I think it is quite a wrong behavior since the messages can have travelled some time before the reception. The sent time and date are given by the "Date:" header and it is set most of the time by the MUA (Mail User Agent, Mutt, Eudora, Thunderbird etc.). imapsync does not touch any byte of messages unless told to do so by an option. Messages on both parts should be identical (some IMAP servers add or even change header lines). Solutions: a) Use the --syncinternaldates option and keep using Eudora. But --syncinternaldates is now turn on by default so if you encounter the issue then the solution is harder, depending on email client softwares and IMAP server softwares. b) use --idatefromheader to set the internal dates on host2 same as the "Date:" headers. It won't work if a) doesn't work. c) In Maildir boxes, after the sync (too late...), use the script learn/adjust_time.pl to change the internal dates from the "Date:" header. (this a Unix fix using touch command) d) Use a better email client or configure it in order it sorts messages by sent date. ======================================================================= Q. imapsync calculates 479 messages in a folder but only transfers 400 messages. What's happen? R1. Unless --useuid is used, imapsync considers a header part of a message to identify a message on both sides. The header part is whole header with "--useheader ALL" or only specific lines depending on --useheader --skipheader or default values. Consequences: 1) Duplicate messages (identical header) are not transferred several times. The result is that you can have more messages on host1 than on host2. R2. With option --useuid imapsync doesn't use headers to identify messages on both sides but it uses their imap uid. In that case duplicates on host1 are transfered on host2. ======================================================================= Q. I need to log every output on a file named log.txt R. Use redirections of both standard and error outputs "> log.txt 2>&1" imapsync ... > log.txt 2>&1 ======================================================================= Q. I need to log every output on a file named log.txt and also to the screen in order to keep seeing what's going on during execution R. Use the tee program (also available on Windows) http://en.wikipedia.org/wiki/Tee_%28command%29 imapsync ... 2>&1 | tee log.txt ======================================================================= Q. I run multiple imapsync applications at the same time then get a warning "imapsync.pid already exists, overwriting it". Is this a potential problem when trying to sync multiple IMAP account in parallel? R1. No issue with the file imapsync.pid if you don't use its content by yourself. This file can help you to manage multiple runs by sending signals to the processes (sigterm or sigkill) using their PID. Each run can have its own pid file with --pidfile option. The file imapsync.pid contains the PID of the imapsync process. This file is removed at the end of a normal run. You can safely ignore the warning if you don't use imapsync.pid file. ======================================================================= Q. Couldn't create [INBOX.Ops/foo/bar]: NO Invalid mailbox name: INBOX.Ops/foo/bar Let begin by an explanation. Example: sep1 = / sep2 = . imapsync reverts each separator automaticaly. a) All / character coming from host1 are converted to . (convert the separator) b) All . character coming from host1 are converted to / (to avoid intermediate unwanted folder creation). So INBOX/Ops.foo.bar (Ops.foo.bar is just one folder name) will be translated to INBOX.Ops/foo/bar Sometimes the sep1 character is not valid on host2 (character "/" usualy) R. Try : --regextrans2 "s,/,X,g" It'll convert / character to X Choose X as you wish: _ or SEP or any string (including the empty string). ======================================================================= Q. The option --subscribe does not seem to work R1. Use it with --subscribed R2. There is also the --subscribe_all option that subscribe to all folders on host2. ======================================================================= Q. Does imapsync retain the \Answered and $Forwarded flags? R. It depends on the destination server. a) If the destination server honors the "PERMAENTFLAGS \*" directive (meaning it accepts any flag) or no PERMAENTFLAGS at all then imapsync synchronises all flags except the \Recent flag (RFC 3501 says about \Recent flag "This flag can not be altered by the client."). b) If the destination server honors the "PERMAENTFLAGS without the special "\*" then imapsync synchronises only the flags listed in PERMANENTFLAGS. Some imap servers have problems with flags not beginning with the backslash character \ (see next question to find a solution to this issue) ======================================================================= Q. How to convert flags? R. use --regexflag For example to convert flag IMPORTANT to flag CANWAIT imapsync ... --regexflag "s/IMPORTANT/CANWAIT/g" --debugflags option --debugflags is usefull to see in details what imapsync does with flags. ======================================================================= Q. How to convert flags with $ to \ character? R. $ and \ are special characters we have to "escape" them. For example to convert flag $label1 to \label1 imapsync ... --regexflag 's/\$label1/\\label1/g' --debugflags ======================================================================= Q. I need to keep only a defind list of flags, how can I do? The destination imap server complains about bad flags (Exchange). R1. Recent imapsync deals with this issue by filter with PERMANENTFLAGS automatically. R2. For example if you want to keep only the following flags \Seen \Answered \Flagged \Deleted \Draft then use these magic --regexflag options (thanks to Phil): --regexflag 's/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg' Analysis is left to the reader. This one is longer and may be use with old perl (no /e regex extension): --regexflag 's/(.*)/$1 jrdH8u/' \ --regexflag 's/.*?(\\Seen|\\Answered|\\Flagged|\\Deleted|\\Draft|jrdH8u)/$1 /g' \ --regexflag 's/(\\Seen|\\Answered|\\Flagged|\\Deleted|\\Draft|jrdH8u) (?!(\\Seen|\\Answered|\\Flagged|\\Deleted|\\Draft|jrdH8u)).*/$1 /g' \ --regexflag 's/jrdH8u *//' ====================================================================== Q. imapsync fails with the following error: flags from : [\Seen NonJunk]["10-Aug-2006 13:00:30 -0400"] Error trying to append string: 58 NO APPEND Invalid flag list R. For some servers, flags have to begin with a \ character. The flag "NonJunk" may be a invalid flag for your server so use for example: imapsync ... --regexflag "s/NonJunk//g" Remark (thanks to Arnt Gulbrandsen): IMAP system flags have to begin with \ character. Any other flag must begin with another character. System flags are just flags defined by an RFC instead of by users. Conclusion, some imap server coders don't read the RFCs (so do I). Recent imapsync deals with this issue by filter with PERMANENTFLAGS automatically. ======================================================================= Q. Flags are not well synchonized. Is it a bug? R. It happens with some servers on the first sync. Also, it was a bug from revision 1.200 to revision 1.207 Solution: run imapsync a second time. imapsync synchronizes flags on each run. ======================================================================= Q. On Unix, some passwords contain * and " characters. Login fails. R. Use a backslash to escape the characters: imapsync ... --password1 \"password\" It works for the star * character, I don't know if it works for the " character. ======================================================================= Q. On Windows, some passwords contain $ characters. Login fails. R1. Enclose passwords between "" imapsync ... --password1 "zzz$zz$$z" R2. Prefix each $ character with a ^ since ^ is the escape character on Windows imapsync ... --password1 zzz^$zz^$^$z For a password that is exactly the 8 characters string $%&<>|^" you have to enter imapsync ... --password1 "$%%&<>|^"^" ======================================================================= Q. With huge account (many messages) when it comes to reading the destination server it comes out this error: "To Folder [INBOX.foobar] Not connected" What can I do? R. May be spending too much time on the source server, the connection timed out on the destination server. Try options --nofoldersizes ======================================================================= Q. imapsync failed with a "word too long" error from the imap server, What can I do? R. Use imapsync release 1.172 or at least 1.166 with options --split1 500 --split2 500 or a old old imapsync (before 1.94) ======================================================================= Q. Does imapsync support IMAP over TLS (IMAPS)? R. Yes natively since release 1.161. still, 2 ways, at least : a) Use --ssl1 and/or --ssl2 options --ssl1 tells imapsync to use ssl on host1 --ssl2 tells imapsync to use ssl on host2 b) Use stunnel http://www.stunnel.org/ Assuming there is an imaps (993) server on imap.foo.org, on your localhost machine (or bar machine) run : stunnel -c -d imap -r imap.foo.org:imaps or using names instead of numbers stunnel -c -d 143 -r imap.foo.org:993 then use imapsync on localhost (or bar machine) imap (143) port. ======================================================================= Q: How to have an imaps server? R. a) Install one b) or use stunnel : Assuming there is an imap (143) server on localhost stunnel -d 993 -r 143 -f c) or use stunnel on inetd imaps stream tcp nowait cyrus /usr/sbin/stunnel -s cyrus -p /etc/ssl/certs/imapd.pem -r localhost:imap2 ======================================================================= Q: I'm trying to use imapsync on win32 for gmail and it requires ssl. Imapsync appears to require IO::Socket::SSL. What can I do? R1: use standalone imapsync.exe, it contains IO::Socket::SSL Perl module embeded. R2: IO::Socket::SSLio-socket-ssl is available on Win32 with Strawberry Perl. ======================================================================= Q: Multiple copies when I run imapsync twice ore more. R1. You can use option --useuid, imapsync then won't use header lines to compare messages in folders. Keep in ming it uses a local cache. imapsync ... --useuid R2. Multiple copies of the emails on the destination server. Some IMAP servers (Domino for example) add some headers for each message transfered. The message is transfered again and again each time you run imapsync. This is bad of course. The explanation is that imapsync considers the message is not the same since headers have changed (one line added) and size too (the header part). You can look at the headers found by imapsync by using the --debug option (and search for the message on both part), Header lines from the source server begin with a "FH:" prefix, Header lines from the destination server begin with a "TH:" prefix. Since --debug is very verbose I suggest to isolate a email in a specific folder in case you want to forward me the output. The way to avoid this problem is by using options --skipheader and --skipsize, like this (avoid headers beginning whith the string "X-"): imapsync ... --skipheader '^X-' --skipsize To skip several headers you can use --skipheader one time imapsync ... --skipheader '^X-|^Status|^Bcc' If you think you have too many header to avoid just use imapsync ... --useheader 'Message-ID' --skipsize Remark. (Trick found by Tomasz Kaczmarski) Option --useheader 'Message-ID' asks the server to send only header lines begining with 'Message-ID'. Some (buggy) servers send the whole header (all lines) instead of the 'Message-ID' line. In that case, a trick to keep the --useheader filtering behavior is to use --skipheader with a negative lookahead pattern : imapsync ... --skipheader '^(?!Message-ID)' --skipsize Read it as "skip every header except Message-ID". ====================================================================== Q. I am transferring mails from one IMAP server to another. I am using an SSL connection. Transferring huge mails (>10MB) takes ages. R. try to transfer the mails without SSL connection. SSL code outside imapsync uses a memory buffer, which gets increased upon reading of mails by 4096 bytes. This creates a huge load on the host imapsync runs on by copying the memory buffers for every 4096 byte step. This does not occur without SSL. (Written by Stefan Schmidt) ====================================================================== Q. What are --subscribe and --subscribed for, and how can they be used? R. In the IMAP protocol each user can subscribe to one or more folders. Then he can configure its email software to just see his subscribed folders list. That's an IMAP feature. Knowing that, the imapsync help says: imapsync --help ... --subscribed : transfers subscribed folders. --subscribe : subscribe to the folders transferred on the host2 that are subscribed on host1. --subscribe_all : subscribe to the folders transferred on the host2 even if they are not subscribed on host1. ====================================================================== Q. I want to exclude a folder hierarchy like "public" R. Use: --exclude '^public\.' or maybe --exclude '^"public\.' In the example given the character "." is the folder separator, you can ommit it. Just take the string as it appears on the imapsync output line : From folders list : [INBOX] [public.dreams] [etc.] ====================================================================== Q. I want to exclude only INBOX R. Use: imapsync ... --exclude '^INBOX$' A good way to see what will be done is to first use: imapsync ... --exclude '^INBOX$' --justfolders --nofoldersizes --dry ====================================================================== Q. I want the --folder 'MyFolder' option be recursive. Two solutions: R1. Use --folderrec 'MyFolder' R2. Use --include '^MyFolder' Then the folder "MyFolder" and all its subfolders will be handled and only them. ====================================================================== Q. How to migrate from or to Exchange 2003 with an admin/authuser account? R. Thomas Edgar wrote the following In case you can glean something from this snippet which allowed us to migrate from Exchange 2003 t0 Cyrus Imap 2.4 (you would reverse the flow to go the other way): imapsync --dry --host1 ExchangeServer.mycompany.com \ --user1 Some.User@mycompany.com --authuser1 ExchangeAdminAccount \ --proxyauth1 --password1 '$pass_with_dollars$' \ --host2 cyrusimapbackend.mycompany.com \ --user2 SameOrDiffererentUser@mycompany.com \ --authuser2 CyrusAdminAccount --proxyauth2 We also needed to: - Ensure the Exchange Admin Acct had IMAP4 enabled in it's profile (it initially didn't!) - Ensure the Some.User Exchange Acct had IMAP4 enabled in it's profile (it initially didn't!) - Add the CyrusAdminAccount to admins line in /etc/imapd.conf - Give CyrusAdminAccount lrswipkxtecda to the Cyrus Imap account being migrated to (- or in your case, from) In case you are not aware: - It will prompt for a password if you don't supply it - the domain part of the fully-qualified email could be omitted in our case - Use --debugimap when testing initial connectivity, if necessary ====================================================================== Q. How to migrate from or to Exchange 2007/2010 with an admin/authuser account? R. The tricks comes from Michele Marcionelli and Benjamin Priestman: This doesn't work: imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ... This might works: imapsync ... --user2 'domain\admin2\user2' --password2 adminpassword2 ... or imapsync ... --user2 'admin2@domain\user2' --password2 adminpassword2 ... where "domain" is set be the user's UPN in Active Directory or the NETBIOS or DNS name of the domain. The exact format might vary depending on local configuration and you should experiment with the different formats. ====================================================================== Q. How to migrate from uw-imap with an admin/authuser account? R. Use --user1="user*admin_user" --password1 "admin_user_password" ====================================================================== Q. How to migrate from cyrus with an admin account? R. Use --authuser1 admin_user ----password1 admin_user_password \ --user1 foo_user --ssl1 In this case, --authmech1 PLAIN will be used by default since it is the only way to go for now. So don't use --authmech1 SOMETHING with --authuser1 admin_user, it will not work. Same behavior with the --authuser2 option. Do not forget the option --ssl1 since PLAIN auth is only supported with ssl encryption most of the time. But it can work without --ssl1 if PLAIN is permitted in normal use. Here is an example: imapsync \ --host1 server1 \ --user1 joe \ --authuser1 AdminAccount \ --password1 AdminAccountPassword \ --ssl1 \ --host2 server2 \ --user2 joe \ --password2 joespassonserver2 \ --exclude '^user\.' ====================================================================== Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet / Netscape servers with an admin account? R: Those imap servers don't allow the typical use of --authuser1 to use an administrative account. They expect the use of an IMAP command called proxyauth that is issued after login in as an administrative account. For example, consider the administrative account 'administrator' and your real user 'real_user'. The IMAP sequence would be: OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT LANGUAGE STARTTLS XSENDER X-NETSCAPE XSERVERINFO AUTH=PLAIN] imap.server IMAP4 service (Sun Java(tm) System Messaging Server ...)) 1 LOGIN administrator password 1 OK User logged in 2 PROXYAUTH real_user 2 OK Completed In imapsync, you can achieve this by using the following options: --host1 source.imap.server \ --user1 real_user \ --authuser1 administrator \ --proxyauth1 \ --passfile admin.txt ====================================================================== Q. Is there anyway of making imapsync purge the destination folder when the source folder is deleted? R. No, that's too dangerous. May be coded in future release. But if the source folder is empty (not deleted) and options --delete2 --expunge2 are used then the destination folder will be empty. ====================================================================== Q. Is it possible to synchronize all messages from one server to another without recreating the folder structure and the target server. R. Yes. For example, to synchronize all messages in all forders on host1 to folder INBOX only on host2: 1) First try (safe mode): imapsync \ ... --regextrans2 's/(.*)/INBOX/' \ --dry --justfolders 2) See if the output says everything you want imapsync to do, --dry option is safe and does nothing real. 3) Remove --dry Check the imap folder tree on the target side, you should only have one: the classical INBOX. 4) Remove --justfolders ====================================================================== Q. I have moved from Braunschweig to Graz, so I would like to have my whole Braunschweig mail sorted into a subfolder INBOX.Braunschweig of my new mail account. R. 1) First try (safe mode): imapsync \ ... --regextrans2 's/INBOX(.*)/INBOX.Braunschweig$1/' \ --dry --justfolders 2) See if the output says everything you want imapsync to do, --dry option is safe and does nothing real. 3) Remove --dry Check the imap folder tree on the target side 4) Remove --justfolders ======================================================================= Q. Give examples about --regextrans2 R. --regextrans2 is used to transform folder names Remember that --regextrans2 applies after the default inversion prefix1 <-> prefix2 and sep1 <-> sep2 Examples: 0) First try with --dry --justfolders options since imapsync shows the transformations it will do without really doing them. Then when happy with the output remove the --dry --justfolders options. 1) To remove INBOX. in the name of destination folders: --regextrans2 's/^INBOX\.(.+)/$1/' 2) To sync a complete account in a subfolder called FOO: a) Seperator is dot character "." and "INBOX" prefixes every folder --regextrans2 's/^INBOX(.*)/INBOX.FOO$1/' or: b) Seperator is slash character "/" and there is no prefix --regextrans2 's#(.*)#FOO/$1#' or: c) Any separator, any prefix solution, FOO is the subfolder: It is a complicated line because every case is taken into account. Type it in one line (or with the \ at the end of first line on Unix shells. --regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \ --regextrans2 's,^INBOX$,${h2_prefix}FOO{h2_sep}INBOX,' 3) to substitute all characters dot "." by underscores "_" --regextrans2 's/\./_/g' 4) to change folder names like this: [mail/Sent Items] -> [Sent] [mail/Test] -> [INBOX/Test] [mail/Test2] -> [INBOX/Test2] --regextrans2 's#^mail/Sent Items$#Sent#' \ --regextrans2 's#^mail/#INBOX/#' ======================================================================= Q. I would like to move emails from InBox to a sub-folder called, say "2010-INBOX" based on the date (Like all emails received in the Year 2010 should be moved to the folder called "2010-INBOX"). R. 2 ways : a) Manually: ------------ 1) You create a folder INBOX.2010-INBOX 2) Mostly every email software allow sorting by date. In INBOX, you select from 1 january to 31 december 2010 messages with the shift key. (in mutt, use ~d) 3) Cut/paste in INBOX.2010-INBOX b) With imapsync: ----------------- imapsync ... \ --search 'SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010' --regextrans2 's/^INBOX$/INBOX.2010-INBOX/' \ --folder INBOX ======================================================================= Q. I want to play with headers line and --regexmess but I want to leave the body as is R. The header/body separation is a blank line so an example: --regexmess 's{\A(.*?(?! ^$))^Date:(.*?)$}{$1Date:$2\nX-Date:$2}gxms' Will replace (HeaderBegin and HeaderEnd are not part of the header) HeaderBegin Message-ID: <499EF800.4030002@blabla.fr> Date: Fri, 20 Feb 2009 19:35:44 +0100 From: Gilles LAMIRAL HeaderEnd by HeaderBegin Message-ID: <499EF800.4030002@blabla.fr> Date: Fri, 20 Feb 2009 19:35:44 +0100 X-Date: Fri, 20 Feb 2009 19:35:44 +0100 From: Gilles LAMIRAL HeaderEnd This example just add an header line "X-Date:" based on "Date:" line. ======================================================================= Q. My imap server does not accept a message and warns "Invalid header". What is the problem? R. You fall in the classical mbox versus Maildir/ format problem. May be you use a misconfigured procmail rule. A header beginning like the following one is in the mbox format, header line 1 has no colon behind "From", header lines 2 through N do have a colon : From foo@yoyo.org Sat Jun 22 01:10:21 2002 Return-Path: Received: ... Any Maildir/ configured imap server may refuse this message since its header is invalid. The first "From " line is not valid. It lacks a colon character ":". To solve this issue you have several solutions a) Remove manually this first "From " line for each message before using imapsync. b) Replace manually the whitespace by a colon in string "From " but you might end with two "From:" lines (just have a look at the other header lines of the message) c) Run imapsync with the following option (this replaces "From "by "From:"): --regexmess 's/\AFrom /From:/' or may be better (no other "From:" collision): d) Run imapsync with the following option (this replaces "From "by "X-om:"): --regexmess 's/\AFrom /X-om:/' e) Run imapsync with the following option (this removes the whole "From " line): --regexmess 's{\AFrom\ [^\n]*(\n)?}{}gxms' Solution e) is solution a) made by imapsync itself. Solutions c) and d) keep "From " lines information (normally it's useless to keep them) Best solutions are e) or d). ======================================================================= Q. The contact folder isn't well copied. How to copy the contact folder? R. Forget the destination server (choose the same) Change the script around line 1426 # ITSD $new_id = $from->copy($t_fold,$f_msg); #$new_id = $to->append_string($t_fold,$string, $flags_f, $d); and tried a copy of the mail instead an append_string. Because we are using the same server, we can use $from->copy Therefore we seem to not download and upload the message and therefore we do not have any format issues. And now it works fine. (Thanks to Hansjoerg.Maurer) ======================================================================= Server specific issues and solutions ======================================================================= ======================================================================= Q. From Zimbra to XXX imapsync ... \ --exclude "Conversation Action Settings" \ --exclude "Quick Step Settings" \ --exclude "News Feed" ======================================================================= Q. From or to HMailServer version 4.4.1. R. You have to add prefix and separator manually because 4.4.1 doesn't honor the NAMESPACE imap command. Example for host1: imapsync ... \ --prefix1 "" --sep1 . No specific option for HMailServer 5.3.3 since NAMESPACE is supported. Maybe --subscribe_all will help you to see all migrated folders. ======================================================================= Q. Synchronising from SmarterMail to XXX imapsync --host1 imap.d1.org --user1 joe --password1 secret1 --sep1 "/" \ --host2 imap.d2.org --user2 joe --password2 secret2 \ --noauthmd5 \ --prefix1 "Inbox/" \ --regextrans2 's#^Inbox$#INBOX#' \ --regextrans2 's#Sent Items$#Sent#' \ --dry --justfolders Maybe add other --regextrans2 to change folder names and see the result. When satisfied, run without --dry --justfolders ======================================================================= Q. Synchronising from XXX to Gmail R. There are some details to get the special [Gmail] sub-folders right. Here's an example of migrating an old "Sent" folder to Gmail's structure: imapsync --host1 mail.oldhost.com \ --user1 my_email@oldhost.com \ --password1 password \ --host2 imap.gmail.com --ssl2 \ --user2 my_email@gmail.com \ --password2 password \ --folder 'INBOX.Sent' \ --regextrans2 's/Sent/Sent Mail/' The same goes for the "All Mail" archive pseudo-folder. ======================================================================= Q. Synchronising from Gmail to XXX R. Gmail needs SSL ./imapsync \ --host1 imap.gmail.com \ --ssl1 \ --authmech1 LOGIN \ --user1 gilles.lamiral@gmail.com \ --password1 gmailsecret \ --host2 localhost --user2 tata \ --password2 tatasecret \ --useheader="X-Gmail-Received" \ --useheader 'Message-Id' If your destination imap server doesn't like "[Gmail]" name, just add option: --regextrans2 's/\[Gmail\]/Gmail/' You can select folders exported to imap within the gmail preferences, unselect all "System labels" ======================================================================= Q. migrate email from gmail to google apps R. Take a look at: http://www.linux-france.org/prj/imapsync_list/msg00639.html http://biasecurities.com/blog/2009/migrate-email-from-gmail-to-google-apps/ http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration/ ======================================================================= Q. Synchronising from Yahoo to XXX R. Use --host1 imap.mail.yahoo.com --sep1 '/' ./imapsync \ --host1 imap.mail.yahoo.com \ --user1 billy \ --password1 secret \ --host2 XXX \ --user2 billy \ --password2 secret \ --sep1 '/' Can also add --ssl1 to gain encrypted transfer from yahoo. SSL seems to be mandatory for yahoo (since november 2011) ======================================================================= Q. from Microsoft's Exchange 2007 to Google Apps for your Domain (GAFYD) R. Take a look at: http://mark.ossdl.de/2009/02/migrating-from-exchange-2007-to-google-apps-mail/ ======================================================================= Q. Syncing from Google Apps domain to Googlemail account A known bug encountered with this output (Alexander is a folder name): ++++ Verifying [Alexander] -> [Alexander] ++++ + NO msg #16 [A96Dh4AwlLVphOAW5MS/eQ:779824] in Alexander + Copying msg #16:779824 to folder Alexander flags from : [\Seen]["04-Jul-2007 14:32:22 +0100"] Couldn't append msg #16 (Subject:[Rieter-Event (please accept with comments)]) to folder Alexander: 46 NO Invalid folder: Sent (Failure) In fact folder "Sent" is just the last folder listed previously as a: ... To Folder [Sent] does not exist yet To Folder [Sonja] Size: 1024546 Messages: 96 ... R. Just run imapsync a time like this : imapsync ... --folder Alexander ======================================================================= Q. I'm migrating from WU to Cyrus, and the mail folders are under /home/user/mail but the tool copies everything in /home/user, how can i avoid that? Two solutions: R. Use imapsync ... --include '^mail' R. or (better) imapsync ... --subscribed --subscribe ======================================================================= Q. I'm migrating from WU to Cyrus, and the mail folders are under /home/user/mail directory. When imapsync creates the folders in the new cyrus imap server, it makes a folder "mail" and below that folder puts all the mail folders the user have in /home/user/mail, i would like to have all those folders directly under INBOX. R. Use imapsync ... --regextrans2 's/^mail/INBOX/' --dry look at the simulation and if all transformations seem good then remove the --dry option. ======================================================================= Q. Migrating from Groupwise to Cyrus R. By Jamie Neil: I eventually managed to get the mail to migrate without errors using the following options: --sep1 / - doesn't report separator so has to be set explicitly. --nosyncacls - doesn't support ACLs. --skipheader '^Content-Type' - MIME separator IDs seem to change every time a mail is accessed so this is required to stop duplicates. --maxage 3650 - some messages just don't seem to want to transfer and produce the perl errors I mentioned before. This prevents the errors, but the bad messages don't transfer. Even though the mail migrated OK, there are a couple of gotchas with Groupwise IMAP: 1) Some of the GW folders are not real folders and are not available to IMAP, the main problem one being "Sent Items". I could find no way of coping the contents of these folders. The nearest I got was to create a "real" folder and copy/move the sent items into it, but imapsync still didn't see the messages (I think because there is something funny about the reported dates/sizes). It think this problem has been rectified in GW6.5. 2) The "skipheader '^Content-Type'" directive is required to stop duplicate messages being created. GW seems to generate this field on the fly for messages that have MIME separators and so it's different every time. 3) Version 6.0.1 of the Groupwise Internet Connector sucks. I was getting server aborts when I pushed it a bit hard! I eventually had to upgrade to 6.0.4 which seems to be a lot more stable. ======================================================================= Q. Migrating from iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) to Groupwise 7.0 I encounter many errors like this: "Error trying to append string: 17847 BAD APPEND" R. GroupWise 7 seems buggy. Apply GroupWise 7 support pack 1 ======================================================================= Q. Migrating from David Tobit V8 R. Use the following options : imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed ======================================================================= Q. Migrating from Tobit David Server 6 ("DvISE Mail Access Server MA-6.60a (0118)") R. Look at the discussion: http://www.linux-france.org/prj/imapsync_list/msg00582.html http://www.linux-france.org/prj/imapsync_list/threads.html#00582 patch saved in ./patches/imapsync-1.337_tobit_V6.patch ======================================================================= Q. I need to migrate 1250 mailboxes, passwords are in a MySQL Database. Can you tell me if your script suits my needs? R. Mailboxes must exist before running imapsync. You have to extract users logins and passwords in a csv file. See the "HUGE MIGRATION" section in the README file. ====================================================================== Q. From Cyrus to Notes Default behavior might works. ====================================================================== Q. From cyrus to dbmail Default behavior might works. ====================================================================== Q: From MailEnable 1.75 R: --sep1 "/" --prefix1 "" Q: From MailEnable 2.2 R: --sep1 "." --prefix1 "" ====================================================================== Q. From GMX IMAP4 StreamProxy R. Use: --prefix1 INBOX and --sep1 . ====================================================================== Q. From Courier to Archiveopteryx R. You can read http://www.archiveopteryx.org/migration/imapsync Default values might be fine now with latest imapsync. ====================================================================== Q. To Sun Java(tm) System Messaging Server 6.2-7.05 Q. To Communigate Pro - Solaris version R. See and run patches/imapsync_1.267_jari ====================================================================== Q. From any to Exchange2007 Several problems: - Big messages: increase the "send- and receive-connector" in exchange2007 to 40 MB. R. 2 solutions R1. With imapsync (only partial success) --skipsize --skipheader 'Received' \ --regexmess 's{\A(.*?(?!^$))^Date:(.*?)$}{$1Date:$2\nReceived: From; $2}gxms' Any user having time to spend to debug Exchange2007 with imapsync is welcome. R2. Other solution Two users succeded by using "MS Transporter Suite" (which is closed expensive nonfree software). ====================================================================== Q. From Microsoft Exchange 2000 IMAP4rev1 server version 6.0.6487.0. R. imapsync ... \ --prefix1 INBOX. ====================================================================== Q: How can I write an .rpm with imapsync R. You'll find an RPM imapsync.spec file in the directory learn/rpm/ It has been downloaded from https://svn.fysik.dtu.dk/projects/rpmbuild/trunk/SPECS/imapsync.spec It has been tested with imapsync 1.434 (May 2011) on CentOS5 and RedHat RHEL5 Linux. (Thanks to Ole Holm Nielsen). This imapsync.spec is derivated from Neil Brown work in 2007. ======================================================================= Q. Where I can read up on the various IMAP RFCs? R. Here: RFC 3501 - INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1 http://www.faqs.org/rfcs/rfc3501.html RFC2683 - IMAP4 Implementation Recommendations http://www.faqs.org/rfcs/rfc2683.html RFC 2595 - Using TLS with IMAP, POP3 and ACAP http://www.faqs.org/rfcs/rfc2595.html RFC 2822 - Internet Message Format http://www.faqs.org/rfcs/rfc2822.html RFC 2342 - IMAP4 Namespace http://www.faqs.org/rfcs/rfc2342.html RFC2180 - IMAP4 Multi-Accessed Mailbox Practice http://www.faqs.org/rfcs/rfc2180.html RFC 4549 - Synchronization Operations for Disconnected IMAP4 Clients http://www.faqs.org/rfcs/rfc4549.html