imapsync/FAQ.d/FAQ.ISP.txt

#!/bin/cat
$Id: FAQ.ISP.txt,v 1.9 2018/05/24 11:34:30 gilles Exp gilles $

This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.ISP.txt

=======================================================================
      Imapsync tips for ISP. Specific issues and solutions.
=======================================================================

* IMAPSync - usage scenario with ISP - by Flávio Zarur Lucarelli.

I thought Id write a quick step by step on my attempts to learn the
imapsync features that matter the most, so it works as we expected in
the cenario in which we use it, which is to migrate customers from
their old ISP to our ISP/email hosting. Thanks to the master Gilles
Lamiral for all his help and hard work.

First of all, remember to use --dry to test things first always and
check the log file to see what would actually happen.

Initially, I used a method where I'd do an exact sync of source to
destination, deleting messages which were in destination, but not
source. I call this "Method 2", below. In this cenario, customer
shouldn't be using the destination account yet. Then, after I switch MX,
I'd do a final sync based on date. The big advantage of this is, you
get an exact sync.

Easier, however, is method 1, where I sync based on message ID, this
way, I can use the same syntax before and after MX change, with no
worries. Only disadvantage, you might not get an exact sync, there
might be some difference in terms of total emails in folders, due to
duplicates, emails that had same message ID in source server.

* Method 1 - sync based on message ID, can use same syntax before and
  after MX change

  imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
		   --addheader

Note: add header adds message ID when it doesn't exist.

This syntax can also be used to sync different source accounts to one
same destination account, simply execute it as many times as desired,
switching source user (user1).

* Method 2 - exact sync source do destination, then sync based on date
  after MX change

My first goal is to have an exact sync of an account from
current/source host to the new/destination host and be able to sync
several times. The --useuid parameter is very important for that
purpose. This is what I use:

  imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
		   --useuid --delete2 --delete2folders 

This makes it so imap.myisp.com (destination) is an exact copy of the
account at imap.gmail.com (source). This is not a problem, since the
user is not using the new host yet. ]You can check Imapsync log files
and surely you will see the final difference should be 0. Check also
for any possible errors in the log (search for "error").

The second goal is to lower the TTL (ex: 5 min) for the host
associated with the MX record, in the domain's DNS server. Let's say
customer has a host mail which his MX points to, with a high TTL
(usually 1 hour). Lower it to 5 min so that, when you change the MX,
it propagates faster.

When comes time to switch over to the new host, do a final sync with
above syntax, before changing the MX. Then, change the MX and tell
your users to start using exclusively the new host.

A few hours after the MX change, we will run Imapsync again. We have
to start preserving emails users move or flag in the new host, which
they started using, so we can't do an exact sync anymore.

The best solution for me was to Sync any new emails (maxage:1) from
source (that could arrive in source even after MX change, due to
cache) and delete such emails from source server. This way, customer's
mailbox is still intact on the source server, except new emails, which
get synced to new server and deleted from source.

  imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
           --folder INBOX --useuid --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1

I personally prefer to keep a copy of users box intact in source, but
if that's not an issue for you, you can remove --folder INBOX and even
--maxage, but then, all emails in source will be deleted. You can use
--maxage 1 with --delete1, however, for all folder (without specifying
--folder INBOX), so only any new email that arrives at source is
copied to destination and deleted from source.

My next goal was to automate the process, so I followed this advice:
http://imapsync.lamiral.info/examples/sync_loop_unix.sh

I also ended up requiring a regex to translate folder names. On the
old server (Gmail), Sent items were in a folder called [Gmail]/E-mails
enviados and on the new server, its simply called SENT. Same with
lixeira (trash) and rascunhos (drafts).

So this was added:

  --regextrans2 "s,\[Gmail\].,," \
  --regextrans2 "s,E-mails enviados,Sent," \
  --regextrans2 "s,Lixeira,Trash," \
  --regextrans2 "s,Rascunhos,Drafts,"

*** Other cenarios

- Sync entire account into 1 folder of another account

  imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \
           --host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
		   --useuid --subfolder2 otheraccountfolder --delete2 --delete2foldersonly /otheraccountfolder/

Above is based on message UID, advantage of no dupes, however, user
must not be using such destination folder in destination account until
you finish syncing.

=======================================================================
=======================================================================