imapsync/FAQ.d/FAQ.SSL_errors.txt

#!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.14 2021/04/19 11:47:44 gilles Exp gilles $

This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt


=======================================================================
               Imapsync SSL errors
=======================================================================

Questions answered in this FAQ are:

Q. What are the errors
  DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
  or
  DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error

Q. What can I do to avoid those "SSL read/write errors"?

Q. SSL connect attempt failed SSL
   routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
   or 
   SSL connect attempt failed SSL
   SSL routines:ssl_choose_client_version:unsupported protocol

Q. How to see the certificate and identify problems in it?

Now the questions again with their answers.

=======================================================================
Q. What are the errors
  DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
  or
  DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error


R1. As they claim, those errors are SSL errors. SSL is not directly
done by imapsync but by an underlying Perl module called
IO::Socket::SSL. Those errors arise sometimes and sometimes
they form a series that ends with imapsync auto-abortion.
Those errors happen with some hosts but not with others,
it's often Exchange or Office365. I don't know what exactly happens.
Those errors happen more often on Windows than on Linux.


=======================================================================
Q. What can I do to avoid those "SSL read/write errors"?

R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
Those errors don't appear with recent releases, post 1.836

R1. Remove all ssl/tls encryption

  imapsync ...   --nossl1 --notls1 --nossl2 --notls2

R2. If you don't want to quit encryption, rerun imapsync until the 
complete sync is over. Those errors are not at the same place 
each time, so imapsync will sync the remaining messages at each run
until none remains.

R3. Run imapsync on a Linux machine, a VM is ok, there are less
    SSL errors on Unix.
    
R4. Use https://imapsync.lamiral.info/X/
    It's a Linux host so response R3 applies there.

R5. Set up a ssltunnel proxy to the host.
    Read the file FAQ.Security.txt for an example to set up
    a ssltunnel proxy.

=======================================================================
Q. SSL connect attempt failed SSL
   routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
   or 
   SSL connect attempt failed SSL
   SSL routines:ssl_choose_client_version:unsupported protocol


R1. Use: 

   imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
   or    
   imapsync ...  --sslargs2 SSL_cipher_list=DEFAULT

   depending on where the error occurs, host1 or host2 or both.

R2. If it doesn't work, I let you try other things, 
    I quote the "SSL_version" section of
    https://metacpan.org/pod/IO::Socket::SSL (Module version: 2.066)

    imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
    imapsync ...  --sslargs1 SSL_version=SSLv2
    imapsync ...  --sslargs1 SSL_version=SSLv23
    imapsync ...  --sslargs1 SSL_version=SSLv3
    imapsync ...  --sslargs1 SSL_version=TLSv1
    imapsync ...  --sslargs1 SSL_version=TLSv1_1
    imapsync ...  --sslargs1 SSL_version=TLSv1_2
    imapsync ...  --sslargs1 SSL_version=TLSv1_3

Those examples are for host1. For host2, use --sslargs2 instead.
Feedback on what worked for you is welcome!

A loop to check every version and print the good ones:

for v in SSLv2 SSLv23 SSLv3 TLSv1 TLSv1_1 TLSv1_2 TLSv1_3; do
        imapsync ...  --sslargs1 SSL_version=$v && GOOD="$GOOD $v"
done
echo "$GOOD"

I reproduce below the documentation of the underlying Perl 
module IO::Socket::SSL used by imapsync:

https://metacpan.org/pod/IO::Socket::SSL
...
SSL_version

Sets the version of the SSL protocol used to transmit data.
'SSLv23' uses a handshake compatible with SSL2.0, SSL3.0 and TLS1.x, 
while 'SSLv2', 'SSLv3', 'TLSv1', 'TLSv1_1', 'TLSv1_2', or 'TLSv1_3' 
restrict handshake and protocol to the specified version. 
All values are case-insensitive. Instead of 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' 
one can also use 'TLSv11', 'TLSv12', and 'TLSv13'.

Support for 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' 
requires recent versions of Net::SSLeay and openssl.

Independent from the handshake format you can limit to set of 
accepted SSL versions by adding !version separated by ':'.
The default SSL_version is 'SSLv23:!SSLv3:!SSLv2' which means, 
that the handshake format is compatible to SSL2.0 and higher, 
but that the successful handshake is limited to TLS1.0 and higher, 
that is no SSL2.0 or SSL3.0 because both of these versions have
serious security issues and should not be used anymore.

You can also use !TLSv1_1 and !TLSv1_2 to 
disable TLS versions 1.1 and 1.2 while still allowing TLS version 1.0.

Setting the version instead to 'TLSv1' might break interaction 
with older clients, which need and SSL2.0 compatible handshake.

On the other side some clients just close the connection 
when they receive a TLS version 1.1 request. 
In this case setting the version 
to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.


=======================================================================
Q. How to see the certificate and identify problems in it?

R. Use the command openssl like this:

  echo | openssl s_client -crlf  -connect imap.gmail.com:993

  echo | openssl s_client -crlf  -connect test1.lamiral.info:993

and examine carefully the content, the  "verify return:" lines,
the chain. Sometimes, the server certificate is ok but not the whole 
chain of certificates so the certification fails.

Here is an example.

One of the certificate is expired:

echo | openssl s_client -crlf -connect test1.lamiral.info:993
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = test1.lamiral.info
verify error:num=10:certificate has expired
notAfter=Apr 11 10:14:05 2021 GMT
verify return:1
depth=0 CN = test1.lamiral.info
notAfter=Apr 11 10:14:05 2021 GMT
verify return:1
---
Certificate chain
 0 s:/CN=test1.lamiral.info
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKjCCBBKgAwIBAgISBHYZCE3qSTIlvq97HI5TpBeAMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAxMTExMDE0MDVaFw0yMTA0MTExMDE0MDVaMB0xGzAZBgNVBAMT
EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAMUTJVdrTl86nDI2yO6Vz5l1qxMMPqJylQcgi9vDHpwsnUq5HGPv+qZNhM69
...

After an complete server update ("apt update && apt upgrade"):

echo | openssl s_client -crlf -connect test1.lamiral.info:993
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = test1.lamiral.info
verify return:1
---
Certificate chain
 0 s:/CN=test1.lamiral.info
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKTCCBBGgAwIBAgISBD4QN3cfB1JpTm75oVrkkAElMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAzMTIxODQxMTJaFw0yMTA2MTAxODQxMTJaMB0xGzAZBgNVBAMT
EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
...


=======================================================================
=======================================================================