News about imapsync and previous releases (back to menu)

imapsync was written on

See ChangeLog to read what has been done in details since 2001.

New features or bugfixes since previous releases:

• 1.678 Improved website! (I hope...)
• Enhancement: Added --sslargs1 and --sslargs2 to pass any ssl parameter for host1 or host2 connection. Example: --sslargs1 SSL_verify_mode=1 --sslargs1 SSL_version=SSLv3
• Enhancement: Added --timeout1 int and --timeout2 int in seconds (--timeout int still available to set both with the same value)
• Enhancement: Added --debugssl int. Default is like --debugssl 1 (Only print out errors).
• Enhancement: Added several polls to know where I shall focus improvements.
• Usability: Added env_proxy call in sub xoauth2() to read proxy settings from environment variable http_proxy without using PERL_LWP_ENV_PROXY=1
• Usability: Wrote TUTORIAL_Unix.
• Usability: Check f1 folder exists when specified by --f1f2 and warns when it doesn't
• Usability: Added --automap --justfolders --dry in imapsync_example.sh and imapsync_example.bat
• Bug fix: Added require Encode::Byte to solve "The locale codeset (cp1252) isn't one that perl can decode" on Win32.
• Refactoring: Removed --allow3xx option.
• Refactoring: Continue to move to one global $sync-> in order to reduce number of parameters in routines.

• 1.670 Folders mapping made easy with --automap
• Enhancement: Added option --automap that guesses folders mapping, for folders like "Sent", "Junk", "Drafts", "All", "Archive", "Flagged".
  • IMAP Special-Use folders described in rfc6154 are supported and also well known names for Exchange and common clients.
  • Automap is turned off by default, to try it a safe method is to use --automap --justautomap, it will exit early
• Enhancement: Added --f1f2 str1=str2 : Force folder str1 to be synced to str2.
  • Option --f1f2 overrides any --automap mapping and any --regextrans2 transformation.
  • Example --f1f2 Spam=Junk to map Spam folder to Junk.
• Enhancement: Added --justautomap to exit after seeing what will happen with --automap and --f1f2 options.
• Enhancement: Added IMAP4 QUOTA values (RFC 2087) when they are available. A warning goes up when it's time to find space.
• Enhancement: Added IMAP4 ID extension.
  • ID feature is on by default if supported by the IMAP servers, use --noid to turn it off.
  • ID Infos returned by the imap servers are shown.
  • Infos sent to the servers are name=imapsync, version=`imapsync --version`, os=$OSNAME`, vendor='Gilles LAMIRAL' and date=`rcs date of release`.
  • For github or other distributors you can hack this part by searching $imapsync_id_github
• Enhancement: Added --logdir path to change the default log directory. Default is LOG_imapsync/
• Usability: Added folders counting in outputs.
• Usability: Better output in the login part
• Usability: Guess prefixes and separators instead of forcing the user to find them.
• Usability: Output, added Host1: or Host2: at the beginning of lines.
• Bug fix: SSL_VERIFY_NONE in --ssl and --tls modes
• Bug fix: Fixed xoauth2 calls. Now XOAUTH2 Works on Windows --authmech1 XOAUTH2 with openssl installed. See FAQ.d/FAQ.XOAUTH2.txt.
• Bug fix: Fixed IO::Socket::SSL constants issue with latest releases
• Bug fix: Changed "Host1: checking all wanted folders exist" not efficient algorithm in order to allow efficiently a 2.4 million folders account. Yes, some accounts have this.
• Refactoring: Started to use global $sync-> in order to reduce number of parameters in routines.
• Refactoring: Some perlcritic fixes.

• 1.644 Mac OS X standalone binary provided!
• Enhancement: Mac OS X standalone binary called imapsync_bin_Darwin
• Enhancement: Added option --subfolder2 SUB Move whole host1 folders hierarchy under folder SUB.
• Enhancement: Added --fetch_hash_set "1:*" to permit Mail2World success. Need a patched Mail::IMAPClient 3.35 in sub fetch_hash()
• Usability: No folders sizes if --justfolders, unless really wanted.
• Usability: No warning about messages when --dry --justfolders are used together.
• Bug fix: Added NOOP in --dry mode during fake APPEND in order to avoid timeouts

• 1.636
• Enhancement: Added errors dump listing at the end, before last folder sizes or the statistics. Turned on by default. Use --noerrorsdump to avoid it.
• Enhancement: Added --maxlinelengthcmd that will be called upon when a line over --maxlinelength value is detected.
• Enhancement: Exchange maxlinelength issue fixed with --maxlinelengthcmd 'reformime -r7' on Linux. Install maildrop package to get reformime command
• Enhancement: Added --testslive. Running imapsync --testslive performs a live test on real accounts (I own).
• Enhancement: Added --pipemess in order to pass all message to an external filter tool like reformime -r7.
• Enhancement: Added xoauth2 support, available on Unix with underlying openssl. Thanks to Joaquin Lopez.
• Enhancement: Added --skipmess regex to skip messages matching a regex. Example --skipmess 'm/[\x80-ff]/' to slip messages with non-7bit characters ( ie a byte somewhere begins with 1 )
• Usability: Better output of folders excluded by --exclude and folders included by --include
• Usability: Added inline help in many places. What you can do with switches to change imapsync behaviour, contextually. Learn imapsync the easy way.
• Bug fix: imapsync --tests should work under any circumstance now, any Unix, Windows, exe or bin or script. 561 non-regression tests
• Bug fix: Options --folderfirst and --folderlast generated an error when their value folder does not exist. Existence is checked.
• Bug fix: Option --disarmreadreceipts used to change Disposition-Notification-To in the body when not available in the header. Now never changes the body in all cases.
• Bug fix: Change default useheader values. Now it is really like --useheader "Message-Id" --useheader "Received". HMailServer replies two lines with --useheader "Message-Id" --useheader "Message-ID" in older imapsync releases.
• Bug fix: in imap_utf7_decode() + character must not be escaped. Was a bug with Cyrillic characters
• Refactoring: Started to split the FAQ file in several parts in FAQ.d/*
• Refactoring: Split the INSTALL file in several parts in INSTALL.d/*

• 1.607 Folders shown also in utf8.
• Caveat to upgrade on Unix!: New Unicode::String Perl module dependency, for utf8 output. See the INSTALL file.
• Enhancement: Added --skipemptyfolders to skip syncing empty host1 folders. They are not created on host2.
• Enhancement: Windows exe now uses IO::SSL 2.002 instead of 1.98 (is it really an enhancement?).
• Usability: Remove /x from regexes applied with --include and --exclude. Blanks no longer have to be explicit with \ or [ ].
• Usability: Added utf8 output for folder names, protocol native utf7imap special characters are not user friendly readable
• Usability: Moved foldersizes output after folders lists output.
• Usability: Added total size transferred after each message copied.
• Usability: After ETA, added number of total to be synced, as a denominator dddd after each message copied, like nnn/dddd where nnn is the number of messages copied. Before there was only nnn without /dddd.
• Bug fix: Cache fix on Windows. File paths with brackets [] are special, they have to be escaped with glob. For example, [Gmail] must be written [[]Gmail[]].
• Bug fix: Added several checks if IsUnconnected. Goal avoid imap commands while disconnected. Reconnexion would be better anyway.

• 1.597 Small things
• Enhancement: Added --nomixfolders to avoid merging folders that are considered different on host1 but the same on destination host2 because of case sensitivities and insensitivities.
• Bug fix: Fixed "imapsync doesn't see created folders in the listing of folder sizes at the end". (second time this bug shows up)

• 1.592 Logging by default! (an internal feature now)
• Caveat to upgrade: New IO::Tee Perl module dependency, for logging. See the INSTALL file.
• Windows upgrade: Win32 binary is now distributed in a zip file with two batches scripts ready to use it.
• Enhancement: Added logging by default in subdirectory ./LOG_imapsync/
• Enhancement: Added option --log turned on by default. Use --nolog to disable logging.
• Enhancement: Added option --logfile. Default is like ./LOG_imapsync/YYYY_MM_DD_HH_MM_SS_user2.txt where YYYY_MM_DD_HH_MM_SS is the launching date and time like 2014_05_20_23_45_10
• Enhancement: Added option --disarmreadreceipts to fix resending read receipts on host2. A must with Exchange as host2.
• Enhancement: Added FTGate support (no EXAMINE imap command used anymore).
• Usability: Updated the code to easily switch on --noreleasecheck by default but allowing --releasecheck (some users want this feature).
• Usability: Warn syncing messges will not be simulated when --dry mode is on and folder on host2 does not exist yet.
• Bug fix: Make TLS works with last IO::Socket::SSL default value (Added SSL_verify_mode => 'SSL_VERIFY_NONE') in --tls mode.
• Bug fix: Fixed bug when creating folders on host2 with --sep2=\\ (Lotus Notes).
• Bug fix: Try to get folder size when a folder on host2 has the same name but with a different case like OLD_MAIL -> old_mail. This avoids "does not exist" output.

• 1.584
• Enhancement: Added --minmaxlinelength to select messages with long lines only. It helps to diagnostic Echange error on messages with lines longer than 9000 characters
• Enhancement: Added --debugmaxlinelength
• Bug fix: --ssl1 --tls2 was buggy because of default SSL_VERIFY_PEER. "Can not go to tls encryption on [localhost]:Unable to start TLS: Cannot determine peer hostname for verification..."
• Bug fix: Check if going to tls is ok, exit otherwise with explicit error message. Thanks to Dennis Schridde for reporting this ugly bug that deserves this CVE-2014-2014.
• Debug: Print separator given by NAMESPACE even when --sep1 oe --sep2 is used.
• Debug: Prints prefix given by NAMESPACE even when --prefix1 or --prefix2 is used. It is a preparation for advising not to use --prefix unless suggested.

Good changes made before, listed because they can help:

• Enhancement: Added --skipcrossduplicates to avoid copying messages that are already copied in another folder, good from Gmail to X.
• Enhancement: Added --debugcrossduplicates to print which messages (UIDs) are skipped with --skipcrossduplicates (and in what other folders they are).
• Enhancement: Added --folderfirst string: sync this folder first. --folderfirst "Work" --folderfirst "Friends"
• Enhancement: Added --folderlast string: sync this folder last. --folderlast "[Gmail]/All Mail" etc.
• Enhancement: Added --maxmessagespersecond to limit messages tranfer rate and fix Office365 throttle limitation (use --maxmessagespersecond 4).
• Enhancement: Added --maxbytespersecond to limit byte transfer rate.
• Enhancement: Added --debugmemory option that prints memory consumption after each message is copied.
• Enhancement: Added --authmech EXTERNAL.
• Enhancement: Adapted behavior for allowing --maxdate --mindate with --noabletosearch. Use internat date instead of Date: header.
• Enhancement: QQMail IMAP4Server success (with --noabletosearch option)
• Usability: Apply same treatment to --delete2duplicates as --delete2. If --uidexpunge2 can be done, do it, else do --expunge2 (unless --nouidexpunge2 or --noexpunge2).
• Usability: Added --timeout in the --help output. Default timeout is 120 seconds now (was nothing by imapsync, 600 with underlying modules).
• Enhancement/Bug fix: Added --ssl1_SSL_version and --ssl2_SSL_version to force the SSL_version in case the default auto-negociation does not work. Example: --ssl1 --ssl1_SSL_version SSLv3
• Enhancement: Added option --messageidnodomain to fix Mailenable bug changing the domain part of Message-Id header, and avoid duplicates
• Enhancement: Added option --syncflagsaftercopy to fix Mailenable bug not getting the flags with the APPEND
• Enhancement: Added option --maxlinelength to skip messages whose max line length is over a number of bytes. Exchange 2013 and Office365 need --maxlinelength 1000 (which is a RFC2822 must) to avoid disconnections.
• Usability: Added --fixInboxINBOX, turned on by default, to map automatically Inbox INBOX folder names.
• Usability: IMAP server removing consecutive spaces on the header part doesn't generate duplicates (MailEnable does that)
• Usability: SSL_verify_mode 0 to avoid warning about Man-In-The-Middle.
• Enhancement: Added XOAUTH authentication. Thanks to Eduardo Bortoluzzi Junior.
• Enhancement: Added --search1 and --search2 to allow different searches on each host.
• Usability: Added --delete1 as an alias for --delete
• Speed: Option --nocheckmessageexists is activated by default since --checkmessageexists often slow down transfers too much.
• Usability: Option --foldersizesatend is on if --foldersizes is on. Off if --nofoldersizesatend
• Usability: Added host2 minus host1 statistic: number of messages and bytes. One difference at the start and one at the end. Useful to detect quickly a difference between host2 and host1. Need both --foldersizes and --foldersizesatend options (which are on by default).
• Usability: Added --foldersizesatend. It gets the folders sizes at the end of the whole transfer. Turned on by default (use --nofoldersizesatend to turn it off).
• Enhancement: Added option --delete2duplicates; it deletes messages in host2 that are duplicates. --delete2duplicates is on when --delete2 is set unless --nodelete2duplicates is set too.
• Enhancement: Added option --pidfilelocking; it aborts imapsync, when just launched, in case another imapsync may be running. Turned off by default
• Usability: Added option --showpasswords. it shows passwords on output instead of "MASKED". Turned off by default. Useful to restart a run by reading the log or to see what happen with special characters.
• Bug fix: Added option --fixslash2 to avoid the 'Invalid mailbox name' error. It acts when --sep1 is character / and --sep2 is not character / and host1 folder name contains --sep2 characters. Turned on by default (--nofixslash2 to unable it).
• Enhancement: Added option --noabletosearch to allow the listing of messages without the imap "SEARCH ALL" command. It's useful for playing with poor imap servers like Softalk 7.6.4 (8.6 is fine with SEARCH ALL).
• Enhancement: Added --noexpungeaftereach to speedup --delete --expunge from Gmail.
• Enhancement: Added option --nocheckmessageexists to speed up with Tobit imap server as host1.
• Enhancement: Added option --exitwhenover option to avoid locking when transfers exceed maximum limit. See for example this Gmail constraint
  • imapsync ... --exitwhenover 2500000000 # 2.5GB if host1 is Gmail
  • imapsync ... --exitwhenover 500000000 # 500MB if host2 is Gmail
• Usability: Default headers used to identify a message are like --useheader "Message-Id" --useheader "Received" (Exchange compatibility bu default).
• Enhancement: Added option --addheader to fix the issue "ignored (no header so we ignore this message)" (on Sent folder usually). Thanks to Ameir Abdeldayem
• Enhancement: Added --nofilterflags to check any STORE flags by ignoring PERMANENTFLAGS (Exchange tests)

This document last modified on ($Id: news.shtml,v 1.10 2016/01/22 01:04:15 gilles Exp gilles $)
Top of the page