imapsync/FAQ.d/FAQ.SSL_errors.txt

#!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.18 2022/01/14 21:20:37 gilles Exp gilles $

This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt


======================================================================
               Imapsync SSL errors
======================================================================

Questions answered in this FAQ are:

Q. What is the error
   DEBUG: .../IO/Socket/SSL.pm:1177: global error: Undefined SSL object

Q. What are the errors
   DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
   or
   DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error

Q. What can I do to avoid those "SSL read/write errors"?

Q. What are the errors
   SSL connect attempt failed SSL
   routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
   or 
   SSL connect attempt failed SSL
   SSL routines:ssl_choose_client_version:unsupported protocol

Q. What is the error
   fatal SSL error: SSL connect attempt failed with unknown error 
   SSL wants a read first

Q. How to see the certificate and identify problems in it?

Now the questions again with their answers.

======================================================================
Q. What is the error
   DEBUG: .../IO/Socket/SSL.pm:1177: global error: Undefined SSL object
   
R. It's a fake error from the Perl Module IO::Socket::SSL
   Imapsync works well despite this fake warning but it's disturbing
   when you encounter errors due to something else, you believe it's
   the issue but no, it's something else to deal with.
   
   This fake error is fixed in  IO::Socket::SSL release 2.073
   https://metacpan.org/dist/IO-Socket-SSL/changes
   "fix #110 - prevent internal error warning in some cases"
   https://github.com/noxxi/p5-io-socket-ssl/issues/110
   
   imapsync.exe release 2.178 uses this fixed 2.073 IO::Socket::SSL

======================================================================
Q. What are the errors
   DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
   or
   DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error


R1. As they claim, those errors are SSL errors. SSL is not directly
done by imapsync but by an underlying Perl module called
IO::Socket::SSL. Those errors arise sometimes and sometimes
they form a series that ends with imapsync auto-abortion.
Those errors happen with some hosts but not with others,
it's often Exchange or Office365. I don't know what exactly happens.
Those errors happen more often on Windows than on Linux.


======================================================================
Q. What can I do to avoid those "SSL read/write errors"?

R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
Those errors appear less often with imapsync releases post 1.836

R1. Remove all ssl/tls encryption

  imapsync ...   --nossl1 --notls1 --nossl2 --notls2

R2. If you don't want to quit encryption, rerun imapsync until the 
complete sync is over. Those errors are not at the same place 
each time, so imapsync will sync the remaining messages at each run
until none remains.

R3. Run imapsync on a Linux machine, a VM is ok, there are less
    SSL errors on Unix.
    
R4. Use https://imapsync.lamiral.info/X/
    It's a Linux host so response R3 applies there.

R5. Set up a ssltunnel proxy to the host.
    Read the file FAQ.Security.txt for an example to set up
    a ssltunnel proxy.

======================================================================
Q. What are the errors
   SSL connect attempt failed SSL
   routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
   or 
   SSL connect attempt failed SSL
   SSL routines:ssl_choose_client_version:unsupported protocol


R1. Use: 

   imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
   or    
   imapsync ...  --sslargs2 SSL_cipher_list=DEFAULT

   depending on where the error occurs, host1 or host2 or both.

R2. If it doesn't work, I let you try other things, 
    I quote the "SSL_version" section of
    https://metacpan.org/pod/IO::Socket::SSL (Module version: 2.066)

    imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
    imapsync ...  --sslargs1 SSL_version=SSLv2
    imapsync ...  --sslargs1 SSL_version=SSLv23
    imapsync ...  --sslargs1 SSL_version=SSLv3
    imapsync ...  --sslargs1 SSL_version=TLSv1
    imapsync ...  --sslargs1 SSL_version=TLSv1_1
    imapsync ...  --sslargs1 SSL_version=TLSv1_2
    imapsync ...  --sslargs1 SSL_version=TLSv1_3

Those examples are for host1. For host2, use --sslargs2 instead.
Feedback on what worked for you is welcome!

A loop to check every version and print the good ones:

for v in SSLv2 SSLv23 SSLv3 TLSv1 TLSv1_1 TLSv1_2 TLSv1_3; do
        imapsync ...  --sslargs1 SSL_version=$v && GOOD="$GOOD $v"
done
echo "$GOOD"

I reproduce below the documentation of the underlying Perl 
module IO::Socket::SSL used by imapsync:

https://metacpan.org/pod/IO::Socket::SSL
...
SSL_version

Sets the version of the SSL protocol used to transmit data.
'SSLv23' uses a handshake compatible with SSL2.0, SSL3.0 and TLS1.x, 
while 'SSLv2', 'SSLv3', 'TLSv1', 'TLSv1_1', 'TLSv1_2', or 'TLSv1_3' 
restrict handshake and protocol to the specified version. 
All values are case-insensitive. Instead of 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' 
one can also use 'TLSv11', 'TLSv12', and 'TLSv13'.

Support for 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' 
requires recent versions of Net::SSLeay and openssl.

Independent from the handshake format you can limit to set of 
accepted SSL versions by adding !version separated by ':'.
The default SSL_version is 'SSLv23:!SSLv3:!SSLv2' which means, 
that the handshake format is compatible to SSL2.0 and higher, 
but that the successful handshake is limited to TLS1.0 and higher, 
that is no SSL2.0 or SSL3.0 because both of these versions have
serious security issues and should not be used anymore.

You can also use !TLSv1_1 and !TLSv1_2 to 
disable TLS versions 1.1 and 1.2 while still allowing TLS version 1.0.

Setting the version instead to 'TLSv1' might break interaction 
with older clients, which need and SSL2.0 compatible handshake.

On the other side some clients just close the connection 
when they receive a TLS version 1.1 request. 
In this case setting the version 
to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.

======================================================================
Q. What is the error
   fatal SSL error: SSL connect attempt failed with unknown error 
   SSL wants a read first

R. If you're using --ssl1 or --ssl2, try instead --tls1 or --tls2

======================================================================
Q. How to see the certificate and identify problems in it?

R. Use the command openssl like this:

  echo | openssl s_client -crlf  -connect imap.gmail.com:993

  echo | openssl s_client -crlf  -connect test1.lamiral.info:993

and examine carefully the content, the  "verify return:" lines,
the chain. Sometimes, the server certificate is ok but not the whole 
chain of certificates so the certification fails.

Here is an example.

One of the certificate is expired:

echo | openssl s_client -crlf -connect test1.lamiral.info:993
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = test1.lamiral.info
verify error:num=10:certificate has expired
notAfter=Apr 11 10:14:05 2021 GMT
verify return:1
depth=0 CN = test1.lamiral.info
notAfter=Apr 11 10:14:05 2021 GMT
verify return:1
---
Certificate chain
 0 s:/CN=test1.lamiral.info
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKjCCBBKgAwIBAgISBHYZCE3qSTIlvq97HI5TpBeAMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAxMTExMDE0MDVaFw0yMTA0MTExMDE0MDVaMB0xGzAZBgNVBAMT
EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAMUTJVdrTl86nDI2yO6Vz5l1qxMMPqJylQcgi9vDHpwsnUq5HGPv+qZNhM69
...


After an complete server update ("apt update && apt upgrade && /etc/init.d/dovecot restart"):

echo | openssl s_client -crlf -connect test1.lamiral.info:993
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = test1.lamiral.info
verify return:1
---
Certificate chain
 0 s:/CN=test1.lamiral.info
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKTCCBBGgAwIBAgISBD4QN3cfB1JpTm75oVrkkAElMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAzMTIxODQxMTJaFw0yMTA2MTAxODQxMTJaMB0xGzAZBgNVBAMT
EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
...

Champagne!

======================================================================
======================================================================