imapsync/FAQ.d/FAQ.Folders_Mapping.txt

#!/bin/cat
$Id: FAQ.Folders_Mapping.txt,v 1.20 2019/07/27 20:04:57 gilles Exp gilles $

This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt



=======================================================================
             Imapsync tips about changing folders names.
=======================================================================

By default, imapsync syncs all folders, one by one, in alphanumeric order.

By default, folders names from host1 are reproduced identical on host2,
except for the prefix and the separator which are automatically adapted
for host2.

The IMAP protocol has a specific way to code folders names,
especially when these names use non-ascii 7bit characters.
This encoding is called utf7imap. 

Imapsync uses the same encoding as IMAP: utf7imap.

In order to well specify folders names on the command line, imapsync
prints the complete folder list of both sides at the beginning of each run.

The left column is the encoding you have to use, without the first enclosing 
square brackets [], the right column is the human utf8 view.

This document gives most examples with the powerful but complex
option --regextrans2. Before using --regextrans2 you should 
consider using --automap and --f1f2 because they are simpler 
to understand and use.

 --automap           : guesses folders mapping, for folders like
                       "Sent", "Junk", "Drafts", "All", "Archive", "Flagged".

 --f1f2    str1=str2 : Force folder str1 to be synced to str2,
                       --f1f2 overrides --automap and --regextrans2.

 --subfolder2   str  : Syncs the whole host1 folders hierarchy under the
                       host2 folder named str.
                       (It does it internally by adding two 
                       --regextrans2 options before all others.
                       Add --debug to see what's really going on.)



Things to know and understand before playing with --regextrans2

*) --regextrans2 is used to transform folders names.

*) --regextrans2 applies after the default 
   inversion prefix1 <-> prefix2 and sep1 <-> sep2.
   So, when elaborating the regex you should focus on
   the right part of the default mapping printed by
   imapsync, the part showing the host2 folder name.
   The section to look at is either the folder size section
   or within the folder loop located "Here" between the lines
++++ Looping on each folder
Here
++++ End looping on each folder
   
*) Several --regextrans2 are possible, they will be applied in the order
   of the command line, each one on the result of the previous one.

*) --regextrans2 uses Perl regex mechanism so it may be hard to master
   this part. It is powerful but not very simple.

*) Windows vs Unix quotes. 

   On windows don't use single quotes ' around the regex string, 
   use double quotes instead, like --regextrans2 "myregex"

   On Linux/Unix use single quotes ' around the regex string, it is
   easier to get what we want with single quotes since the shell 
   won't change the inner string. Use --regextrans2 'myregex'

*) Good method to elaborate any --regextrans2 string

   First, elaborate the --regextrans2 string with --dry --justfolders options. 

     imapsync ... --dry --justfolders

   With --dry imapsync shows the transformations it will do without 
   really doing them, --dry is the "do nothing" mode.
   With --justfolders imapsync will work only with folders, 
   messages won't be taken into account, so it will be fast
   and focused on the folders names.

   When the output shows what you expect imapsync to do with folders
   names, you can remove the --dry option. Keep the --justfolders 
   option in order to see if the destination server host2 accepts
   to create the folders.

   When everything is ok with folders you might remove --justfolders,
   imapsync will also transfer messages.
   Showing folders sizes is good then transferring messages, it allows
   ETA calculation and it's a supplementary check on folders.

=======================================================================
Q. Give examples about --regextrans2


Examples: 

1) To remove INBOX. in the name of destination folders

  imapsync ... --regextrans2 's/^INBOX\.(.+)/$1/'

2) To change only INBOX to Inbox_Migrated

   imapsync ... --regextrans2 's{^INBOX$}{Inbox_Migrated}'


2a) To sync all folders to INBOX

  imapsync ... --regextrans2 "s/.*/INBOX/"


2b) To sync a complete account in a subfolder called FOO

With imapsync release 1.641 and former, simply use

  imapsync ... --subfolder2 FOO
  
Examples to this subfolder problem for any release
can be seen below, in another Q/R section.


3) to substitute all characters dot "." by underscores "_"
    --regextrans2 "s,\.,_,g"

3b) to substitute all doublequotes " by underscores _

On Linux/Unix:

  --regextrans2 's,\",_,g' 

On Windows:

  --regextrans2 s,\^",_,g

3c) to substitute all characters * % . and # by underscores _

On Linux/Unix:

  --regextrans2 'tr,*%.#,_,'

On Windows:

  --regextrans2 "tr,*%.#,_,"

You can increase the *%.# list by any unwanted character,
plus, the order doesn't count. See 3d) for &

3d) It is a bad idea to substitute & characters since &
is a character to encode non-ascii characters in IMAP folder names.


4) to change folder names like this:
[mail/Sent Items] -> [Sent]
[mail/Test]       -> [INBOX/Test]
[mail/Test2]      -> [INBOX/Test2]

On Linux/Unix:

    --regextrans2 's,^mail/Sent Items$,Sent,' \
    --regextrans2 's,^mail/,INBOX/,'


======================================================================
Q. Is it possible to synchronize all messages from one server to 
another without recreating the folder structure and the target server.

R. Yes.

For example, to synchronize all messages in all folders on host1
to folder INBOX only on host2:

1) First try (safe mode):

  --regextrans2 "s/.*/INBOX/" --dry --justfolders

2) See if the output says everything you want imapsync to do, 
   --dry option is safe and does nothing real.
   
3) Remove --dry 
   Check the imap folder tree on the target side, you should
   only have one: the classical INBOX.

4) Remove --justfolders


=======================================================================
Q. I would like to move emails from InBox to a sub-folder called,
   say "2010-INBOX" based on the date. Like all emails received in the
   year 2010 should be moved to the folder called "2010-INBOX".

R. 2 ways :

a) With imapsync:
-----------------

  imapsync ... \
    --search "SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010"
    --f1f2 "INBOX=2010-INBOX" \
    --folder INBOX

See also the FAQ FAQ.Messages_Selection.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
to get more examples about how to select messages.

b) Manually:
------------

1) You create a folder INBOX.2010-INBOX

2) Mostly every email software allow sorting by date. In INBOX, you
   select from 1 January to 31 December 2010 messages with the shift key.
   (in mutt, use ~d)

3) Cut/paste in INBOX.2010-INBOX


=======================================================================
Q. How to sync a complete account in a subfolder called FOO,
   the hard way.

R0. See the easy above using --subfolder2

R1. Several ways to do it.

  a) Separator is dot character "." and "INBOX" prefixes every folder

On Linux/Unix:

    --regextrans2 's,^INBOX(.*),INBOX.FOO$1,'

On Windows:

    --regextrans2 "s,^INBOX(.*),INBOX.FOO$1,"

  or:
  
  b) Separator is the slash character "/" and there is no prefix 

On Linux/Unix:

   --regextrans2 's,(.*),FOO/$1,'
   
   or with doublequotes
   
   --regextrans2 "s,(.*),FOO/\$1,"

On Windows:

   --regextrans2 "s,(.*),FOO/$1,"
   
   or:

  c) Any separator, any prefix solution, FOO is the subfolder:

  It is a complicated line because every case is taken into account.
  Type it in one line (or with the \ at the end of first line on Unix shells.

On Linux/Unix:

   --regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \
   --regextrans2 's,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,'

On Windows:

  --regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," ^
  --regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,"

  
=======================================================================
=======================================================================