This commit adds a try-except block around the IMAP ID command to catch any exceptions that may occur during its execution. If an exception is raised, it will log a warning message with the details of the error, improving the error reporting and allowing users to understand what went wrong without crashing the application.
When `starttls = yes` is configured and the server does not advertise
STARTTLS in its capability list, the previous code silently returned
without attempting TLS, leaving the connection in cleartext.
RFC 2595 section 9 explicitly documents this attack vector:
"A man-in-the-middle attacker can remove STARTTLS from the
capability list or generate a failure response to the STARTTLS
command."
Silently skipping STARTTLS in that case makes offlineimap completely
vulnerable to such capability-stripping attacks: the attacker wins by
simply removing the STARTTLS keyword from the server greeting.
Fix: when STARTTLS is not advertised but is configured, emit a warning
and attempt STARTTLS anyway. If the server genuinely does not support
it, imaplib will raise an error and the existing error-handling path
raises OfflineImapError, aborting the connection. If the attempt
succeeds, TLS is established normally.
The resulting behaviour is:
- Server advertises STARTTLS → unchanged, works as before.
- Server does NOT advertise STARTTLS (possible MITM strip):
- Warning is logged to alert the user.
- STARTTLS is attempted regardless.
- If the server rejects it: OfflineImapError, connection aborted.
- If the server accepts it: TLS is established, sync continues.
Reported by: hartwork
References: RFC 2595 §9
- Implement tolerance for non-standard IMAP servers that do not provide reliable CAPABILITY after STARTTLS.
- Introduce configuration option 'allow_nonstandard_capabilities' to enable fallback using pre-TLS capabilities.
Before this, authentication will fail after the access token has
expired, and if this is handled by an external program using
oauth2_access_token_eval, offlineimap3 will be able to renew the token
by itself. We should allow oauth2_access_token_eval to run in this case.
One might imagine that authentication needs to be reperformed for
example because of bad internet, and that the token will therefore be
refreshed unnecessarily. I believe most users of
oauth2_access_token_eval will use caching and that this will not be a
big problem.
Signed-off-by: Noa Torstensvik <noa@torstensvik.se>
We introduce a separate closing flag that's checked in
acquireconnection, decoupling the connectionlock from the semaphore
such that they can be acquired separately.
Signed-off-by: Michael Hohmuth <hohmuth@sax.de>
This commit addresses issues where certain IMAP servers (like Protonmail Bridge)
abruptly close the socket during the authentication phase.
Following a simplified approach:
1. Updated __authn_helper to initiate STARTTLS unconditionally at the
beginning of the loop.
2. Modified acquireconnection to catch OfflineImapError. If the socket is
detected as dead during authentication, it now logs a warning and retries
the entire connection process from scratch (up to 3 attempts).
3. This ensures each retry starts with a fresh connection and all configured
authentication methods available, avoiding permanent list modification.
Certain IMAP servers (like Protonmail Bridge) suffer from bugs where they hang and drop the connection when receiving specific authentication commands (such as `AUTHENTICATE PLAIN` immediately after STARTTLS). Previously, if the socket died during an authentication attempt, offlineimap would abort the entire synchronization process immediately.
This commit introduces `AuthMethodSocketDeadError` to handle this scenario gracefully. When an authentication mechanism kills the socket, it is temporarily removed from the list of available mechanisms for that session. Offlineimap then reconnects and retries the remaining authentication mechanisms (e.g., falling back from PLAIN to LOGIN), allowing synchronization to continue successfully without entering an infinite loop.
ç
When a user sets `starttls = yes`, offlineimap expects to negotiate a secure TLS tunnel. However, if a MITM attacker intercepts the connection and removes the `STARTTLS` capability from the server's greeting, the `if 'STARTTLS' in imapobj.capabilities` check fails silently. This causes offlineimap to skip TLS negotiation and proceed to send credentials in plaintext.
This commit introduces a strict check in `__start_tls()`: if the user requested STARTTLS but the server does not advertise it, offlineimap will immediately abort with a clear error message, protecting the user's password from being exposed. To connect insecurely, the user must explicitly set `starttls = no`.
This patch adds a health check for pooled IMAP connections in the acquireconnection() method. After acquiring a connection from the pool, it sends a NOOP command to verify that the connection is still alive. If the NOOP command fails, it logs the failure, cleans up the stale connection, and recursively calls acquireconnection() to get a new one. This helps ensure that clients receive healthy connections from the pool and reduces the likelihood of encountering errors due to stale connections.
This patch adds a helper function `_is_socket_alive` that checks if the underlying socket connection of an IMAP object is still usable. It uses `select` to check for readability and errors on the socket, and also checks if the socket has been closed by the peer. This function is called before attempting authentication methods to avoid long timeouts if the socket is already dead (e.g., after a failed STARTTLS). If the socket is found to be broken, it logs a debug message and aborts further authentication attempts.
This commit removes the redundant shutdown() calls in start_tls() and acquireconnection() that were added to work around a TLS connection failure. The shutdown() calls were causing spurious warnings in the cleanup chain when the socket was already closed. The TLS connection failure should now be handled by the exception raised in start_tls() without needing to shutdown the socket again.
This avoid problems with the socket being closed twice and generating warnings, while still properly handling the TLS connection failure. The exception raised in start_tls() will indicate the failure without needing to shutdown the socket again.
This patch works around imaplib2's unconditional CAPABILITY call after the TLS handshake, which causes a 60-second hang with servers like Protonmail Bridge that do not respond to CAPABILITY at that stage. The workaround temporarily substitutes a no-op for imaplib2's _get_capabilities method during the handshake, allowing it to set _tls_established=True without triggering the problematic CAPABILITY call. The patch also guards against a redundant second shutdown() call on the same socket, which can raise [Errno 9] Bad file descriptor and generate spurious warnings in the cleanup chain.
In various cases we were passing string messages, which is incompatible
with how ui.error handles its argument, leading to bugs like #233Fix#233
Signed-off-by: serge-sans-paille <sergesanspaille@free.fr>
Here a simple way that allows to migrate password to system keyrgin, and
use it to run offlineimap.
Signed-off-by: Kirill A. Korinsky <kirill@korins.ky>
This patch enables the ID extension of IMAPv4.
The patch sends the client name and the client version to the server.
Usually, the server doesn't require it, but in some cases the server
drop de connection if the ID is not send.
#Close #71
This patch sets the environment variable OFFLINEIMAPSYNCMODE to either
full, quick or idle depending on the context of the pre- and
postsynchook.
Adding the context as an argument was considered but this would break
existing configurations and it makes calling a program directly more
cumbersome. Some programs (e.g. imapfilter) may not know what to do with
this extra argument.
Signed-off-by: Mart Lubbers <mart@martlubbers.net>
Up to a4863b2 offlineimap did not include a default CA bundle. And
folks who set cert_fingerprint (because they might connect to a
host using a self-signed cert or an onion service without the onion
address in the SANs) were able to validate their certificates.
Since a4863b2 you always have a `sslcacertfile` configured (since
it always falls back to the os one) and thus the old way didn't
work anymore.
If a use defines a `cert_fingerprint` there is not much use to
validate the cert through the CA chain, since the fingerprint
is the stronges verification you can get. Therefor we can disable
verfication when `cert_fingerprint` is set.
This enables users to fetch emails again from onion services or
hosts using self-signed certifcates, but doesn't question nor
change any other behavior.
Fixes#41
When ui is set to 'Curses Blinkenlights' and debug logs are enabled,
we get an exception with 'embedded null character'.
Remove the NULL from the log, keeping the log message same as before.
Signed-off-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
The current code mixed string and bytes leading to:
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'honk.sigxcpu.org'
sequence item 1: expected str instance, int found
Traceback:
File "/usr/share/offlineimap3/offlineimap/accounts.py", line 298, in syncrunner
self.__sync()
File "/usr/share/offlineimap3/offlineimap/accounts.py", line 374, in __sync
remoterepos.getfolders()
File "/usr/share/offlineimap3/offlineimap/repository/IMAP.py", line 648, in getfolders
imapobj = self.imapserver.acquireconnection()
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 592, in acquireconnection
self.__authn_helper(imapobj)
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 449, in __authn_helper
if func(imapobj):
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 362, in __authn_gssapi
imapobj.authenticate('GSSAPI', self.__gsshandler)
File "/usr/lib/python3/dist-packages/imaplib2.py", line 691, in authenticate
typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
File "/usr/lib/python3/dist-packages/imaplib2.py", line 1684, in _simple_command
return self._command_complete(self._command(name, *args), kw)
File "/usr/lib/python3/dist-packages/imaplib2.py", line 1404, in _command
literal = literator(data, rqb)
File "/usr/lib/python3/dist-packages/imaplib2.py", line 2247, in process
ret = self.mech(self.decode(data))
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 318, in __gsshandler
reply = ''.join(reply)
Closes: #46
Signed-off-by: Guido Günther <agx@sigxcpu.org>
In case of an exception in XOAUTH2 code refresh (HTTP Error 400: Bad
Request), following exception occurs:
ERROR: While attempting to sync account 'xxx'
__init__() missing 3 required positional arguments: 'msg', 'hdrs', and 'fp'
Traceback:
File ".../offlineimap3/offlineimap/accounts.py", line 298, in syncrunner
self.__sync()
File ".../offlineimap3/offlineimap/accounts.py", line 374, in __sync
remoterepos.getfolders()
File ".../offlineimap3/offlineimap/repository/IMAP.py", line 648, in getfolders
imapobj = self.imapserver.acquireconnection()
File ".../offlineimap3/offlineimap/imapserver.py", line 591, in acquireconnection
self.__authn_helper(imapobj)
File ".../offlineimap3/offlineimap/imapserver.py", line 448, in __authn_helper
if func(imapobj):
File ".../offlineimap3/offlineimap/imapserver.py", line 382, in __authn_xoauth2
imapobj.authenticate('XOAUTH2', self.__xoauth2handler)
File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 682, in authenticate
typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 1675, in _simple_command
return self._command_complete(self._command(name, *args), kw)
File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 1395, in _command
literal = literator(data, rqb)
File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 2238, in process
ret = self.mech(self.decode(data))
File ".../offlineimap3/offlineimap/imapserver.py", line 257, in __xoauth2handler
raise type(e)(msg, exc_info()[2])
The exception 'e' is of type HTTPError, which does not have the same kind of
constructor as normal Python exceptions.
Instead, print the constructed message and just raise the existing
exception.
With that change, the same condition triggers another problem further on:
ERROR: While attempting to sync account 'xxx'
tuple index out of range
Traceback:
File ".../offlineimap3/offlineimap/accounts.py", line 298, in syncrunner
self.__sync()
File ".../offlineimap3/offlineimap/accounts.py", line 374, in __sync
remoterepos.getfolders()
File ".../offlineimap3/offlineimap/repository/IMAP.py", line 648, in getfolders
imapobj = self.imapserver.acquireconnection()
File ".../offlineimap3/offlineimap/imapserver.py", line 664, in acquireconnection
elif isinstance(e, socket.error) and e.args[0] == errno.ECONNREFUSED:
because e.args is empty.
When I ported offlineimap from Python 2 to Python 3 I removed the Debug for
IMAP (imaplib2).
The reason was offlineimap was setting the Debug directly in imaplib2,
not using the proper way (using the IMAP4 argument). Because we are
removing the virtual_imaplib2, I removed this option.
I removed this line in offlineimap/init.py:303:
---8<---
300 dtype = dtype.strip()
301 self.ui.add_debug(dtype)
302 if dtype.lower() == u'imap':
-303 imaplib.Debug = 5
304
305 if options.runonce:
306 # Must kill the possible default option.
---8<---
With this patch, the debug level 5 is restored in imaplib if the user
set the -d ALL or -d imap in offlineimap.
Server capabilities are returned as list of bytes. We need convert them
to list of strings.
This patch do it. Probably you must recreate your cache after this patch.
Call to OfflineImapError has the arguments:
- message
- severity
- error code
The None argument ir wrong here, we can remove it and then we have three arguments not four.
This patch updates the cram-md5 auth. We include two steps:
- Convert the password variable from string to bytes. This change is
because in Python2 strings and bytes are the same, but not in Python3
- Updates the call to hmac.new, now the digestmod argument is mandatory.
I used hashlib.md5, because we need md5 hash.
Closes#19
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
Error when using the XOAUTH2 token refresh logic:
("POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str. (configuration is: {....}", <traceback object at 0x7fc6e69f2b40>)
[' File ".../offlineimap3/offlineimap/accounts.py", line 298, in syncrunner
self.__sync()
', ' File ".../offlineimap3/offlineimap/accounts.py", line 374, in __sync
remoterepos.getfolders()
', ' File ".../offlineimap3/offlineimap/repository/IMAP.py", line 446, in getfolders
imapobj = self.imapserver.acquireconnection()
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 579, in acquireconnection
self.__authn_helper(imapobj)
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 443, in __authn_helper
if func(imapobj):
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 377, in __authn_xoauth2
imapobj.authenticate(\'XOAUTH2\', self.__xoauth2handler)
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 681, in authenticate
typ, dat = self._simple_command(\'AUTHENTICATE\', mechanism.upper())
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 1674, in _simple_command
return self._command_complete(self._command(name, *args), kw)
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 1394, in _command
literal = literator(data, rqb)
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 2237, in process
ret = self.mech(self.decode(data))
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 253, in __xoauth2handler
raise type(e)(msg, exc_info()[2])
']
Fix by encoding the data passed to urllib.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
Rodolfo García Peñas (kix)
Noa Torstensvik
Michael Hohmuth
Derek Schrock
serge-sans-paille
lilydjwg
Magarnicle
Kirill A. Korinsky
Nicolas Sebrecht
Mart Lubbers
mh
Sudip Mukherjee
Guido Günther
Thomas De Schampheleire