So far, the :tag command in the notmuch and jmap workers allows adding a
tag (by prefixing it with '+') and removing it (by prefixing it with
'-'). Add a new functionality to this command, allowing it to toggle a
tag by prefixing it with '!'.
Fixes: https://todo.sr.ht/~rjarry/aerc/292
Changelog-added: It is now possible to toggle notmuch and JMAP tags.
Signed-off-by: inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
When requesting the bodyStructure email property, the spec says that the
default returned body part properties are: ["partId", "blobId", "size",
"name", "type", "charset", "disposition", "cid", "language",
"location"].
Specifically, the "subParts" property is *NOT* expected by default.
Fastmail servers seem to use a different default "bodyProperties" list
and implicitly return "subParts" even if not requested.
Other JMAP server implementations (e.g. Apache James) do use the RFC
default "bodyProperties" and therefore omit returning "subParts" unless
explicitly asked to.
Change the requested "bodyProperties" to include "subParts" as well.
Aerc needs them to display messages.
NB: for later, we should probably change our message abstraction not to
include any body structure. This makes very little sense to expose that
to users. In fact, aerc has code to explicitly prevent users from
selecting multipart/* parts. Not requesting that information to the
server would make it easier.
Link: https://datatracker.ietf.org/doc/html/rfc8621#section-4.2
Reported-by: Benoit Tellier <btellier@linagora.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Tim Culverhouse <tim@timculverhouse.com>
The jmap:// outgoing backend always copies (actually, just tags) the
sent emails with the mailbox that has the "sent" role. Regardless of the
copy-to or :send -t <folder> argument. Only the copy-to-replied setting
is effective.
Change the CopyTo parameter of the StartSendingMessage backend operation
to hold a list of folder names.
In the JMAP worker, label the sent message with the list of folders,
eliminating duplicates (e.g. do not label with "sent" role twice).
Reported-by: Matěj Cepl <mcepl@cepl.eu>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Matěj Cepl <mcepl@cepl.eu>
Cache threads that we have fetched. Monitor threads for changes, and
fetch any email which we don't have for any thread that changes.
Changelog-added: The JMAP backend now supports full thread fetching
and caching (limited within a single mailbox).
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
A folders contents need refreshing if we don't have any saved state, or
if the sort criteria is different than the cache, or if the filter is
different than the cache. The previous logic required a refresh if
the cached filter == nil. This is almost always the case: opening a
plain folder has a nil filter. The only filter criteria is added in the
parsing step (ie: inMailbox). This meant every directory change that
wasn't already filtered required a Refresh. The new logic fixes this.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Track the list of updated mailboxes and make a second request which
updates only those mailboxes who require an update. The queryChanges
call can be expensive. In brief testing of the previous logic (which
queried changes for all mailboxes), the request time was reduced from
~300ms to ~60ms. Reducing the number of queryChanges calls has a
dramatic performance improvement here. A typical operation of moving
messages from one mailbox to another will result in only two mailboxes
being updated - which is a much faster request than the entire list.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Directly check if the updated email belongs to our selected mailbox.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The JMAP worker list directories logic performed a few too many
requests. The logic was:
1. Query the remote server for the mailbox state
2. If the state matches our cache, use the cache
3. If it doesn't match, query for the mailbox list
However, we can rely on the update channel to send updates for the
mailbox list, as long as we have a list to be updated, ie the cached
state value is not the empty string. Modify the logic such that the
cache is always trusted, and updates will be received from the update
channel. Make a request for the mailbox list only when there is nothing
in the cache, or we have missing mailboxes to fetch.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Start listening for remote changes as soon as we have the connect
message. The previous implementation listened once a ListDirectories
call was received, unnecessarily delaying updates.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The JMAP worker receives all updates from the update channel. This can
cause some UI responsiveness issues when the update is slow. For a
common task of updating message keywords, we can update the UI
immediately if the response indicates a successful set call.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Ignore and log a charset error when fetching a message part.
Reported-by: Matěj Cepl <mcepl@cepl.eu>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a new models.UID type (an alias to string). Replace all occurrences
of uint32 being used as message UID or thread UID with models.UID.
Update all workers to only expose models.UID values and deal with the
conversion internally. Only IMAP needs to convert these to uint32. All
other backends already use plain strings as message identifiers, in
which case no conversion is even needed.
The directory tree implementation needed to be heavily refactored in
order to accommodate thread UID not being usable as a list index.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Inwit <inwit@sindominio.net>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
When a push notification arrives, automatically fetch any newly created
messages. Set the Recent flag on these messages to trigger a
notification in the UI.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Refactor JMAP thread fetching to occur in a single request. We use a
result reference method call on Email/get to fetch the message info for
each message in a thread.
Previously, we checked the cache and only fetched information for
messages we didn't already have. This results in two requests and is
typically slower for most contexts. Also, in most cases if we are
fetching an email we don't already have it's because we also need to
fetch the entire thread. The only case this wouldn't happen is if we get
a new email in a thread. We can optimize this by fetching message info
for created messages in the push method at a later time.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Fetch Identities only when required: when we are sending an email.
Refactor connect.go to check the state string of the Session on every
request, and update the session as needed. Move the Session validity
check to happen on any request, eg when our client returns an error we
first update the Session object (via re-authenticating) and then retry
the request.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a new copy-to-replied setting in accounts.conf to copy sent replies
to the same folder than their replied message.
Requested-by: Tristan Partin <tristan@partin.io>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tristan Partin <tristan@partin.io>
Fetch an email's entire thread in the JMAP backend.
Changelog-added: Fetch entire threads in the JMAP backend.
Signed-off-by: Tristan Partin <tristan@partin.io>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
If a session is found in the cache, check that it works by issuing
a GetIdentities request. If that request fails, invalidate the cache and
go through the authentication.
Ensure that the session is valid and explicitly fail if it is not.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Tim Culverhouse <tim@timculverhouse.com>
This reverts commits:
9e93d9efdb ("jmap: fix go static check failure")
0465509eed ("jmap: skip Email/get call if no emails to get")
9f97c698e3 ("jmap: fetch entire threads")
Issues have been reported about disappearing sent messages.
Reported-by: Tristan Partin <tristan@partin.io>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tristan Partin <tristan@partin.io>
Since the update to go-jmap 0.5.0, the IsAscending property is now taken
into account even if it is false.
The initial JMAP implementation wrongfully assumed the behaviour and
inverted the sort -r logic. Fix that so that messages are ordered as
expected.
Link: https://git.sr.ht/~rockorager/go-jmap/commit/1db959676c10398a8
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
The missing variable was being set but unused.
Fixes: 9f97c698e3 ("jmap: fetch entire threads")
Signed-off-by: Tristan Partin <tristan@partin.io>
Acked-by: Robin Jarry <robin@jarry.cc>
A v1 got committed when a v2 existed.
Fixes: f9113810cc ("jmap: invalidate cache if mailbox state is not consistent")
Signed-off-by: Tristan Partin <tristan@partin.io>
Acked-by: Robin Jarry <robin@jarry.cc>
JMAP was missing good thread support, especially when compared to Gmail.
This will fetch entire threads when possible.
Signed-off-by: Tristan Partin <tristan@partin.io>
We weren't checking if the cached state was the same as the remote state
before reading it. This led to aerc not knowing about new mailboxes on
the remote.
Signed-off-by: Tristan Partin <tristan@partin.io>
Acked-by: Robin Jarry <robin@jarry.cc>
JMAP is able to automatically determine sender and recipients based on
the message headers after it is submitted for sending. However this
means that it is not possible to send a message with the From header not
matching the account with this approach (or to send the message to
recipients not listed in the headers). Luckily, JMAP allows setting the
envelope containing the envelope sender and recipients manually. Modify
the code to do so.
Also bump go-jmap to include a fix needed for this to work.
Signed-off-by: Karel Balej <balejk@matfyz.cz>
Acked-by: Robin Jarry <robin@jarry.cc>
Allow the backend workers to combine the filter terms. Currently, the
consecutive filters are joined in the message store with a space (" ").
This works well for most backends, but makes the filter combination for
notmuch confusing.
Example:
Issuing two consecutive filter commands in notmuch
:filter not tag:list
:filter tag:list
would create the following filter query
'not tag:list tag:list'
This is not what users would expect; they expect:
'(not tag:list) and (tag:list)'
Note that the notmuch backend works correctly for the given query, but
produced a query that does not match the user's expectation. This patch
fixes this.
The combination of filter terms in other backends remains the same.
Reported-by: Ángel Castañeda <angel@acsq.me>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Support the draft flag wherever flags are used. Automatically set it
when postponing a message, and allow recalling a message without the -f
flag if it has the draft flag set, regardless of what folder it's in.
Notmuch doesn't seem to pick up on the draft flag when indexing even
though the flag is set on the maildir file. Explicitly set all tags
corresponding to set flags when appending a message in notmuch.
Changelog-added: Support the `draft` flag.
Signed-off-by: Jason Cox <me@jasoncarloscox.com>
Acked-by: Robin Jarry <robin@jarry.cc>
For some reason, a JMAP server may send message details with empty
blobId values. This is invalid but it should not cause a crash.
Error: runtime error: slice bounds out of range [-2:]
goroutine 16 [running]:
git.sr.ht/~rjarry/aerc/worker/jmap/cache.(*JMAPCache).blobPath()
git.sr.ht/~rjarry/aerc/worker/jmap/cache/blob.go:43 +0x95
git.sr.ht/~rjarry/aerc/worker/jmap/cache.(*JMAPCache).GetBlob()
git.sr.ht/~rjarry/aerc/worker/jmap/cache/blob.go:11 +0x18
git.sr.ht/~rjarry/aerc/worker/jmap.(*JMAPWorker).handleFetchMessageBodyPart()
git.sr.ht/~rjarry/aerc/worker/jmap/fetch.go:116 +0x26f
git.sr.ht/~rjarry/aerc/worker/jmap.(*JMAPWorker).handleMessage()
git.sr.ht/~rjarry/aerc/worker/jmap/worker.go:142 +0x25f
git.sr.ht/~rjarry/aerc/worker/jmap.(*JMAPWorker).Run()
git.sr.ht/~rjarry/aerc/worker/jmap/worker.go:177 +0x105
git.sr.ht/~rjarry/aerc/app.NewAccountView.func3()
git.sr.ht/~rjarry/aerc/app/account.go:105 +0x57
created by git.sr.ht/~rjarry/aerc/app.NewAccountView in goroutine 1
git.sr.ht/~rjarry/aerc/app/account.go:98 +0x468
Ignore a blobId when it is an empty string.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Tristan Partin <tristan@partin.io>
When encountering a statement such as:
go functionName()
The identifier is not a local symbol but should be looked up in the
current package. Do not consider that all these statements will refer to
local variables.
The only way to do this is to run a second analyzer that depends on the
first one. Store all unresolved methods and functions into
a indirectCalls struct. Reuse that result in the second analyzer to
resolve the function declarations and check their bodies for the
defer log.PanicHandler() statement.
Fix one newly reported issues.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Tim Culverhouse <tim@timculverhouse.com>
Define a SearchCriteria structure. Update the FetchDirectoryContents,
FetchDirectoryThreaded and SearchDirectory worker messages to include
this SearchCriteria structure instead of a []string slice.
Parse the search arguments in a single place into a SearchCriteria
structure and use it to search/filter via the message store.
Update all workers to use that new API. Clarify the man page indicating
that notmuch supports searching with aerc's syntax and also with notmuch
specific syntax.
getopt is no longer needed, remove it from go.mod.
NB: to support more complex search filters in JMAP, we need to use an
email.Filter interface. Since GOB does not support encoding/decoding
interfaces, store the raw SearchCriteria and []SortCriterion values in
the cached FolderContents. Translate them to JMAP API objects when
sending an email.Query request to the server.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Tested-by: Inwit <inwit@sindominio.net>
Avoid importing code from worker/lib into lib. It should only be the
other way around. Move the message parsing code used by maildir,
notmuch, mbox and the eml viewer into a lib/rfc822 package.
Adapt imports accordingly.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Tested-by: Inwit <inwit@sindominio.net>
github.com/mitchellh/go-homedir has not received any update since 2019.
The last release of github.com/kyoh86/xdg was in 2020 and it has been
marked as deprecated by its author.
Replace these with internal functions.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Moritz Poldrack <moritz@poldrack.dev>
The gob encoder requires registration of types used during encoding.
There are several types defined in the Session object that don't
directly or indirectly get registered with gob. As a result, the session
object never actually gets cached, requiring an authentication step
which is often unnecessary.
Use json encoding for this object to provide a simpler serialization
path.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Moritz Poldrack <moritz@poldrack.dev>
Fix the following error when opening another aerc instance with the same
jmap account:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x9a1ffd]
git.sr.ht/~rjarry/aerc/worker/jmap/cache.(*JMAPCache).get(0x99d08e?, {0xbc3c1a?, 0xc00003a160?})
git.sr.ht/~rjarry/aerc/worker/jmap/cache/cache.go:47 +0x1d
git.sr.ht/~rjarry/aerc/worker/jmap/cache.(*JMAPCache).GetSession(0xc00052a030?)
git.sr.ht/~rjarry/aerc/worker/jmap/cache/session.go:8 +0x29
git.sr.ht/~rjarry/aerc/worker/jmap.(*JMAPWorker).handleConnect(0xc00055e180, 0x0?)
git.sr.ht/~rjarry/aerc/worker/jmap/connect.go:29 +0xd3
git.sr.ht/~rjarry/aerc/worker/jmap.(*JMAPWorker).handleMessage(0xc000311500?, {0xcc8b00?, 0xc0001fcff0?})
git.sr.ht/~rjarry/aerc/worker/jmap/worker.go:114 +0x9f
git.sr.ht/~rjarry/aerc/worker/jmap.(*JMAPWorker).Run(0xc00055e180)
git.sr.ht/~rjarry/aerc/worker/jmap/worker.go:177 +0x10c
git.sr.ht/~rjarry/aerc/widgets.NewAccountView.func3()
git.sr.ht/~rjarry/aerc/widgets/account.go:110 +0x65
created by git.sr.ht/~rjarry/aerc/widgets.NewAccountView
git.sr.ht/~rjarry/aerc/widgets/account.go:103 +0x518
Do not return an error if the leveldb cannot be opened, log a message
and fallback on the in-memory cache.
Fixes: be0bfc1ae2 ("worker: add jmap support")
Reported-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
When archiving a message, the counts of archive folder are changed.
When use-labels=true, the archive folder is hidden and should remain so.
Do not send updates to the UI.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Add a WorkerInteractor interface. Avoid exposing any public fields in
the types.Worker.
This will set the stage to implement a middleware pattern for the
workers, i.e. to map folder names between the ui and the backend.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Add support for JMAP backends. This is on par with IMAP features with
some additions specific to JMAP:
* tagging
* sending emails
This makes use of git.sr.ht/~rockorager/go-jmap for the low level
interaction with the JMAP server. The transport is JSON over HTTPS.
For now, only oauthbearer with token is supported. If this proves
useful, we may need to file for an official three-legged oauth support
at JMAP providers.
I have tested most features and this seems to be reliable. There are
some quirks with the use-labels option. Especially when moving and
deleting messages from the "All mail" virtual folder (see aerc-jmap(5)).
Overall, the user experience is nice and there are a lot less background
updates issues than with IMAP (damn IDLE mode hanging after restoring
from sleep).
I know that not everyone has access to a JMAP provider. For those
interested, there are at least these two commercial offerings:
https://www.fastmail.com/https://www.topicbox.com/
And, if you host your own mail, you can use a JMAP capable server:
https://stalw.art/jmap/https://www.cyrusimap.org/imap/download/installation/http/jmap.html
Link: https://www.rfc-editor.org/rfc/rfc8620.html
Link: https://www.rfc-editor.org/rfc/rfc8621.html
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
