Go has evolved significantly over the years and has introduced some
handy helper functions that make the code easier to read.
Use helper functions like slices.Contains, map.Copy, and
strings.CutPrefix, when appropriate.
Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
The various handleFetchXYZ functions do not consistently request
X-GM-LABELS when interacting with GMail, and the in-memory "label state"
can easily get out-of-sync with the server.
This patch fixes this, and also ensures that we only log that we're
attaching a label to a message when we actually do (and if so, puts the
account name in the log line).
Signed-off-by: Simon Martin <simon@nasilyan.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Use the GMail IMAP extension to fetch labels for messages when
use-gmail-ext is enabled. Labels are automatically fetched when fetching
headers for a message and when fetching flags for a message. They should
be correctly cached.
Link: https://developers.google.com/workspace/gmail/imap/imap-extensions
Changelog-added: Readonly support for GMail labels.
Signed-off-by: Jan Černohorský <jan@grsc.cz>
Tested-by: Vojtěch Káně <vojta001@vkane.cz>
Acked-by: Robin Jarry <robin@jarry.cc>
I noticed that the fix I made via b576ab28 (and the general idea to get
rid of materialized RUE counters) does not work, because we're extremely
unlikely to have all the messages' flags in memory. So the patch worked
"by chance" when updating messages that are in the active message list
(e.g. their headers are already fetched) but not for "out of view"
messages.
This alternative patch "just" captures unsolicited MessageUpdate
messages from the server, and updates the Unseen counter according to
the status reported by the server for that message. This allows to
synchronize our view with the server's.
Fixes: b576ab28 ("properly update UI upon message (un)read in [...]")
Fixes: https://todo.sr.ht/~rjarry/aerc/307
Signed-off-by: Simon Martin <simon@nasilyan.com>
Tested-by: Karel Balej <balejk@matfyz.cz>
Acked-by: Robin Jarry <robin@jarry.cc>
So far, the :tag command in the notmuch and jmap workers allows adding a
tag (by prefixing it with '+') and removing it (by prefixing it with
'-'). Add a new functionality to this command, allowing it to toggle a
tag by prefixing it with '!'.
Fixes: https://todo.sr.ht/~rjarry/aerc/292
Changelog-added: It is now possible to toggle notmuch and JMAP tags.
Signed-off-by: inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Threaded directories were never marked as loaded and thus the terminal
bell was never rung when a new message arrived in them. Add the missing
directive to mark them as loaded when appropriate.
Fixes: 8d4704775b ("msgstore: do not trigger notification on initial load")
Changelog-fixed: The new-message bell is now rung again for threaded
directories as well.
Cc: Remko Tronçon <remko@el-tramo.be>
Signed-off-by: Karel Balej <balejk@matfyz.cz>
Acked-by: Robin Jarry <robin@jarry.cc>
There currently is no difference between updating the store of
a directory that was already loaded, and a directory that has not yet
been initialized. When a directory is opened, and the initial update
request is processed, all messages are therefore treated as new, and the
directory change notification (i.e. bell) is triggered.
To prevent triggering this update, track whether the directory was
already initialized, and if not, don't send out any change
notifications.
Signed-off-by: Remko Tronçon <remko@el-tramo.be>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a new models.UID type (an alias to string). Replace all occurrences
of uint32 being used as message UID or thread UID with models.UID.
Update all workers to only expose models.UID values and deal with the
conversion internally. Only IMAP needs to convert these to uint32. All
other backends already use plain strings as message identifiers, in
which case no conversion is even needed.
The directory tree implementation needed to be heavily refactored in
order to accommodate thread UID not being usable as a list index.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Inwit <inwit@sindominio.net>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Extract a function to configure the message store from its constructor
to reconfigure the store when the data changes.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Usually, a MUA sets a flag for messages that have been forwarded, but
this is currently not the case for aerc. Consider the forwarded flag
and prepare aerc to set it everytime the :forward command is called and
ends successfully.
Changelog-added: The :forward command now sets the forwarded flag.
Signed-off-by: inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a new copy-to-replied setting in accounts.conf to copy sent replies
to the same folder than their replied message.
Requested-by: Tristan Partin <tristan@partin.io>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tristan Partin <tristan@partin.io>
Avoid eternal spinner on the message list when the imap server
advertises some message UIDs but fails to provide their headers when
aerc asks from them.
When an error occurs, or if some UIDs are not returned, make sure to
report the errors to the message list UI.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Tristan Partin <tristan@partin.io>
If no match were found in the References and In-Reply-To headers,
allow threading by looking at subjects.
This behaviour is disabled by default. Add a setting to enable it.
Changelog-added: Allow fallback to threading by subject with
`[ui].threading-by-subject`.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Matěj Cepl <mcepl@cepl.eu>
In 24eab0f5ef the condition under which the triggerDirectoryChange
callback is executed was changed so that it would now only run if there
were new messages in the currently scrolled into view part of the
message list whereas previously it ran for new messages anywhere in the
list.
The motivation for this was to make the mail-received hook work even
when the tab of the receiving account is not currently focused by
fetching the new message's/s' headers needed for the hook to execute
based on what changed in the scrolled into part of the list. This
limitation is needed because apparently we don't currently have a better
way to recognize new messages other than comparing the list of UIDs
provided by the worker and that kept by the store.
My current understanding is that we cannot remove the scroll view limit
because then we would be fetching headers for all messages in the
directory which can be a somewhat intensive operation. However we can
disregard the view limit for the bell as that does not require the
headers: it only cares about that something has changed.
It is noteworthy that the bell is rung on aerc startup: I assume that
this occurs with the initial load of the message store for every account
when the else branch where directoryChange is set to true inevitably
executes.
Overall, this patch is more of a workaround than a proper fix: the ideal
situation would be if we were able to fetch headers for new messages
independently of the scroll status. However as this is how this was
before, it should be suitable as a temporary solution.
There are also further problems here: currently we have triggerNewEmail
and triggerDirectoryChange callbacks which are both supposed to run when
new mail is received, the latter only implicitly. And yet they both use
a different mechanism and thus execute under different circumstances. It
would be ideal to move the bell into the new mail trigger and get rid of
the directory change one as it is otherwise unused. However this is not
done here because for some setups the new mail trigger does not run
until the tab is focused which is what the aforementioned commit was
meant to fix but apparently succeeded in doing so only partially. The
directoryChange trigger does not have this drawback and thus should be
kept until the issue is resolved for all setups.
Also note that for instance for O365 mail the bell still doesn't work
properly as there new mail only appears in the store after the given
folder is reopened (:prev-folder :next-folder) and the bell is thus rung
only then.
Fixes: 24eab0f5ef ("msglist: fetch headers even when not focused")
Fixes: https://todo.sr.ht/~rjarry/aerc/249
Changelog-fixed: Restore previous behaviour of the new message bell
which was broken in the last two releases for at least some setups.
Signed-off-by: Karel Balej <balejk@matfyz.cz>
Acked-by: Robin Jarry <robin@jarry.cc>
A single notmuch message can represent multiple files. As a result,
file-based operations like move, copy, and delete can be ambiguous. Add
a new account config option, multi-file-strategy, to tell aerc how to
handle these ambiguous cases. Also add options to relevant commands to
set the multi-file strategy on a per-invocation basis.
If no multi-file strategy is set, refuse to take file-based actions on
multi-file messages. This default behavior is mostly the same as aerc's
previous behavior, but a bit stricter in some cases which previously
tried to be smart about multi-file operations (e.g., move and delete).
Applying multi-file strategies to cross-account copy and move operations
is not implemented. These operations will proceed as they have in the
past -- aerc will copy/move a single file. However, for cross-account
move operations, aerc will refuse to delete multiple files to prevent
data loss as not all of the files are added to the destination account.
See the changes to aerc-notmuch(5) for details on the currently
supported multi-file strategies.
Changelog-added: Tell aerc how to handle file-based operations
on multi-file notmuch messages with the account config option
`multi-file-strategy` and the `-m` flag to `:archive`, `:copy`,
`:delete`, and `:move`.
Signed-off-by: Jason Cox <me@jasoncarloscox.com>
Tested-by: Maarten Aertsen <maarten@nlnetlabs.nl>
Acked-by: Robin Jarry <robin@jarry.cc>
When split view and threading mode are enabled, the message is blinking.
First, since commit ddfa5cac1f ("msgstore: fix deadlock in thread
builder"), the threadCallback is never set to nil: the thread builder
calls it continuously.
After setting it to nil, the message is still blinking once. To avoid this,
don't call the onSelect method (which points to
AccountView.updateSplitView()) from the thread builder: the message is
already displayed.
Let's rewrite Select and selectPriv:
- Select(): it takes the lock and calls the onSelect callback;
- selectPriv: the assumption is that the lock is held. It doesn't call
the onSelect callback. This function is only used by the thread builder.
Fixes: 588be1a284 ("store: improve cursor position")
Fixes: ddfa5cac1f ("msgstore: fix deadlock in thread builder")
Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: Robin Jarry <robin@jarry.cc>
When appending a message to a folder, always create the destination
folder if it doesn't exist.
Signed-off-by: Johannes Thyssen Tishman <johannes@thyssentishman.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The mail-added hook runs whenever a message is added to a folder.
Note that the hook does not run when a new message is received (the
mail-received hook already covers that) but instead runs whenever aerc
itself adds a message to a folder, e.g. when moving or copying a
message.
Changelog-added: `mail-added` hook that triggers when a message is added
to a folder.
References: https://todo.sr.ht/~rjarry/aerc/136
Signed-off-by: Jason Cox <me@jasoncarloscox.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The mail-deleted hook runs whenever a message is deleted from a folder.
Note that this means moving a message from one folder to another
triggers the mail-deleted hook.
Changelog-added: `mail-deleted` hook that triggers when a message is
removed/moved from a folder.
References: https://todo.sr.ht/~rjarry/aerc/136
Signed-off-by: Jason Cox <me@jasoncarloscox.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a toggle option (-t) to :fold/:unfold commands to allow for
switching the folding status of a thread.
Changelog-Added: Toggle folding with `:fold -t`.
Signed-Off-By: inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Tested-by: Jason Cox <me@jasoncarloscox.com>
Extend the :fold/:unfold behaviour to allow for multiple folding levels.
Signed-Off-By: inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Define a SearchCriteria structure. Update the FetchDirectoryContents,
FetchDirectoryThreaded and SearchDirectory worker messages to include
this SearchCriteria structure instead of a []string slice.
Parse the search arguments in a single place into a SearchCriteria
structure and use it to search/filter via the message store.
Update all workers to use that new API. Clarify the man page indicating
that notmuch supports searching with aerc's syntax and also with notmuch
specific syntax.
getopt is no longer needed, remove it from go.mod.
NB: to support more complex search filters in JMAP, we need to use an
email.Filter interface. Since GOB does not support encoding/decoding
interfaces, store the raw SearchCriteria and []SortCriterion values in
the cached FolderContents. Translate them to JMAP API objects when
sending an email.Query request to the server.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Tested-by: Inwit <inwit@sindominio.net>
Add a UI config value to enable showing of "thread-context", similar
to `notmuch show --entire-thread=true`. Add an associated style called
"msglist_thread_context" which can be used to style such messages.
Currently this feature is only supported by notmuch. It would be
possible for maildir to implement as well, IMAP with gmail custom
extensions, and JMAP. This patch merely implements the notmuch version
and puts the groundwork in for handling these sorts of displays.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
When using notmuch and changing labels, there is no reselection logic as
there is when using a :mv command for other backends. This results in
any label change that removes the message from the current query in
having no selection. Add some simple reselection logic in the Sort
callback so any call to Sort (which will refresh the message list)
ensures the previously selected message is still selected, and if it
isn't in the store anymore we select the same index of message as was
previously selected.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
Do not rely on MessageList.Draw only to fetch missing headers. In Draw,
report the current scroll offset and length to the message store and use
them to determine if a new message UID should be candidate for fetching
headers.
This allows the mail-received hook to work even when the message list is
not focused.
Fixes: https://todo.sr.ht/~rjarry/aerc/147
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tristan Partin <tristan@partin.io>
Store the association between uids and threads in a map instead of just
having the threads in a slice. This simplifies the lookup of a thread
when we have an uid and we can avoid computationally expensive tree
walks.
The threadbuilder will rebuild the uids from the given thread structure.
Hence there is no need now to keep a threads slice around.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Tested-by: inwit <inwit@sindominio.net>
The msgstore keeps a map of UIDs it has requested headers for. The map
is only cleared of pending headers when either an error or a valid
header is received. This can lead to pending headers not being removed
from the list (and therefore never re-requested) if a user has navigated
away from the directory before the response is received.
Delete the pending headers list if the request is finished or cancelled.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a Context field to requests which we may want to cancel when
changing directories.
Add a Cancelled meta-message to inform the UI that a request was
cancelled (as opposed to Done or Error). Delete callbacks when a request
is Cancelled.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Most, if not all, requests to the worker that may need cancellation are
sent by the msgstore. These requests are typically only relevant when
the msgstore is the selected one. Pass a context from the dirlister to
the msgstore when it is selected. This context will be passed in future
commits to worker requests. The context is cancelled when a new
directory is selected.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Allocate properly sized maps when receiving a DirectoryContents or
DirectoryThreaded message
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Every message exchanged between the UI and the worker(s) is logged at
least twice with some obscure IDs and struct pointer names.
When enabling trace logging, this floods the files for very little or no
benefit and makes the meaningful logging message hard to catch.
Also, other messages tend to flood the output for no specific reason.
These are most of the time leftovers from the development process and
are not useful for tracking potential issues.
Remove these.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Inwit <inwit@sindominio.net>
There are several ways the ui message channel can fill up leading to
deadlocks.
1) Invalidate() changes the value of uiState to DIRTY. The following
call sequence:
QueueRedraw()
Invalidate()
QueueRedraw()
Leads to multiple nil messages being queued in the message channel
whereas one could assume that the second QueueRedraw() would do
nothing. This is caused by the tri-state nature of uiState.
2) We use the same channel to convey state change, keyboard events and
redraw requests. Since a keyboard event almost always triggers
a redraw, we end up trying to append a redraw message in the same
goroutine that reads from the channel. This triggers a deadlock when
there are more than 50 pending messages.
Solve the issue by using multiple channels, one per type of message that
needs to be sent to the main ui thread.
Remove QueueRedraw() and merge its functionality in Invalidate(). Only
use a DIRTY/CLEAN state to determine if something needs to be queued in
the redraw channel.
Use a channel for quitting instead of an atomic. Restructure some code
functions to have a cleaner API.
Use a for loop in the main thread and select from all channels.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Maarten van Gompel <proycon@anaproy.nl>
The msgstore trims the uid list when a message was deleted. It does so
whether the message is in the message store or not. If the message was
already deleted and a MessagesDeleted is received, the store will be
trimmed regardless. This can cause spurious emails to be removed from
the msgstore due to a race condition with the fs watcher / imap update
channel. When a message is deleted, the UI will received two
MessagesDeleted and in some cases trim the list, removing too many
emails.
Fix the handling of deleted messages so that the uid list is rebuilt of
all messages except the deleted ones specifically.
Reported-by: Drew Devault <sir@cmpwn.com>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Inwit <inwit@sindominio.net>
Store the Directory RUE counts on the Directory data model. Use
DirectoryInfo messages to update the Directory model. Access Directories
via the dirlist instead of via the msgstore. Remove unused fields on
DirectoryInfo, all backends now give accurate counts.
Move refetch logic into dirlist
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry<robin@jarry.cc>
Create msgstores when a types.Directory message is received. This
removes a quirk from the IMAP worker that msgstores are created on the
first DirectoryInfo, and updated on the second. This path requires three
messages in order to get an updated message store. By creating from
types.Directory, we ensure that any subsequent DirectoryInfo can be
routed to a msgstore.
Remove the field DirInfo from the msgstore initializer, it isn't needed
at initialization and isn't available with this refactor.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Use the Backend interface to report Backend capabilities. Previously,
these were reported via a DirectoryInfo message, however they have
nothing to do with a directory and should be reported directly by the
backend. Add Capabilities method to Backend interface, satisfy this in
each backend, and use it on the UI side.
Remove Caps field from DirectoryInfo
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry<robin@jarry.cc>
Fetching the message list is done in a convoluted way. The UI receives a
DirectoryInfo message, which creates a message store. It then receives a
second DirectoryInfo (an oddity from the IMAP worker), and this
DirectoryInfo is passed to the message store which then requests a fetch
of the message list via store.Sort.
Use the OpenDirectory done response to tell the message store to fetch
messages. This makes the code easier to follow, and does not rely on
quirks from the IMAP worker.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Deprecate triggers and replace them with hooks. Now that aerc supports
running arbitrary ex commands over IPC, it is possible to run internal
aerc commands *and* shell commands via external shell scripts. Hooks
only allow running shell commands. Hooks info is passed via environment
variables.
Implements: https://todo.sr.ht/~rjarry/aerc/136
Signed-off-by: Moritz Poldrack <git@moritz.sh>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Using a list of integers is not optimal. Use a bit mask instead.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
When scrolling while the thread builder is running, aerc freezes. This
issue can be easily reproduced by keeping the down arrow pressed while
a folder is loading with local threading enabled.
This is caused by the threadCallback function calling store.Select which
acquires threadsMutex. However, threadCallback is already called with
threadsMutex acquired, causing a deadlock.
Fix the issue by adding a new selectPriv function that does not acquire
the lock and call this one in threadCallback *and* store.Select. Do not
reset threadCallback to nil as it was before.
Fixes: 6b8e0b19d3 ("split: refactor to prevent stuck splits")
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Thomas Vigouroux <me@vigoux.giize.com>
Refactor split logic (again...) to prevent stuck splits. Use callback
from msgstore.Select to tell the split which message to display. This
keeps the account from having to track displayed messages, which
prevents race conditions in certain situations.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
The addition of the iterator factory added a thread builder when using
server side threads. A conditional for returning UIDs when selecting
messages would check for a not-nil store.builder, and return UIDs from
the thread builder. When using server side threads, there was no
mechanism to update the threads after a message deletion or move, so
store.Uids() would return a stale set of UIDs. This would allow the user
to "select" a deleted email (the cursor would disappear).
Add an update mechanism for the threads if server side threads are
enabled. When building thread UIDs, check for deleted or hidden threads.
Fixes: https://todo.sr.ht/~rjarry/aerc/123
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Inwit <inwit@sindominio.net>
