mirror of
https://github.com/atuinsh/atuin.git
synced 2026-10-26 10:15:43 +01:00
taylor.fish
8359eb0b31
This PR unifies the location of all Atuin data into the `~/.atuin` directory that we already create. New installations will store data in `~/.atuin/data` rather than `~/.local/share/atuin`, and config items like `config.toml` and `themes` directly in `~/.atuin` rather than in `~/.config/atuin/`. Atuin will continue to use the old locations if the new ones don't exist. This is done on a granular[^1] basis; if `~/.atuin/data` doesn't exist, Atuin will load data from `~/.local/share/atuin`, even if `~/.atuin` contains other items like `config.toml` or `themes`. Existing installations will continue to work as-is and will not be automatically migrated. If you wish to migrate your installation, [follow these steps](#migrating-existing-installations). The location of `~/.atuin` can be customized with the `ATUIN_HOME` environment variable. It must be an absolute path; otherwise it will be ignored. If you use `ATUIN_HOME` to switch between multiple different profiles, you should set [`daemon.socket_path`] explicitly in each, to avoid connecting to the wrong daemon. [^1]: Generally, each *top-level* item is looked up separately. For example, you can have `~/.atuin/themes` and `~/.config/atuin/config.toml` simultaneously, but if you have `~/.atuin/themes/a.toml` and `~/.config/atuin/themes/b.toml`, Atuin will only load `~/.atuin/themes` and won't find `b.toml`. There is an exception for AI-related items (`permissions.ai.toml`, `skills`, and `TERMINAL.md`): they're looked up in `~/.atuin` only if `~/.atuin/config.toml` exists, because older installations may have created those same files but with a different meaning (scoped to `$HOME` rather than global). For more information, see the [migration instructions](#migrating-existing-installations). [`daemon.socket_path`]: https://docs.atuin.sh/latest/configuration/config/#socket_path ### Why make this change? As Atuin continues to gain new features that require managing ever-growing types and amounts of data, we feel that storing everything in a single directory is the best way to ensure users get a predictable, reliable experience across all system types. Using a single directory makes it easier for users to locate all Atuin-related data on their system, back up or migrate that data to a new system, and ensure that every running instance of Atuin is accessing the same configuration and databases (a concern when the daemon is started in a different environment from the client). It also makes it easier to completely uninstall Atuin, which, although we hope our users never feel the need to do, is something we believe should be easy and straightforward. The introduction of `ATUIN_HOME` in this PR also makes it easier to use multiple independent Atuin “profiles”, each with their own data and config.[^2] This is very difficult to accomplish currently: setting `XDG_CONFIG_HOME` and `XDG_DATA_HOME` mostly works, but those environment variables get propagated to child processes and will be inherited by, for example, a shell spawned by the pty-proxy. There are some Atuin-specific variables like `ATUIN_CONFIG_DIR`, but they only affect the location of some items. For some background, Atuin's current file layout is inconsistent and nonstandard. Although Atuin is sometimes described as following the [XDG Base Directory Specification][xdg], it is only partially compliant. Although Atuin uses `$XDG_CONFIG_HOME` and `$XDG_DATA_HOME`, it: * Stores logs in `~/.atuin/logs`, rather than `$XDG_STATE_HOME`. * Stores the `atuin` binary in `~/.atuin/bin`, rather than `~/.local/bin`, when installed from the install script. * Stores sockets in `/tmp/atuin-$UID`, rather than `$XDG_RUNTIME_DIR`. * Stored the dotfiles cache in `$XDG_DATA_HOME` rather than `$XDG_CACHE_HOME`. * Stores pidfiles and file locks in `$XDG_DATA_HOME` rather than `$XDG_RUNTIME_DIR`. * Uses XDG Base Directories on Windows and macOS, where doing so is not standard. [xdg]: https://specifications.freedesktop.org/basedir/latest/ [^2]: If you do this, you should set [`daemon.socket_path`] in each profile's `config.toml` to avoid connecting to the wrong daemon. ### Why not use the “correct” paths? There are several reasons why “just using the correct paths” isn't so straightforward: * The XDG Base Directory Specification is primarily a Linux specification. On Windows, it is atypical and nonstandard to follow the XDG Base Directory Specification; rather, directories like `%USERPROFILE%\AppData\Local` should be used. Aside from being potentially quite unintuitive to our users, Windows does not seem to have a convention for distinguishing between cache, config, and data directories; in [`dirs`], for example, [`cache_dir`], [`config_local_dir`], and [`data_local_dir`] all return the exact same path on Windows. This leaves issues waiting to happen if we store a file with the same name in the config and data directories, which would be different on Linux but the same on Windows. The situation on macOS isn't much better. In practice, some command-line tools follow the XDG Base Directory Specification on macOS, but there is no consensus on whether this is correct, as macOS has its own official directories for application data, such as `~/Library/Application Support` and `~/Library/Caches`. As with Windows, macOS's set of user directories does not map cleanly onto the XDG Base Directory Specification; [`config_local_dir`] and [`data_local_dir`] return the same path on macOS, thus risking the same issue with files having the same name. It is worth noting that even if these issues were to be resolved, and Atuin used the “correct” directories for each operating system type without conflicts, this would require providing three sets of instructions for any operation that touches Atuin's files: rather than “edit `~/.atuin/config.toml`”, we would have to say “edit `~/.config/atuin/config.toml` on Linux, `~/Library/Application Support/atuin/config.toml` on macOS, or `~\AppData\Local\atuin\config.toml` on Windows”. Providing a consistent experience across systems is one of the main reasons we've decided to move to `~/.atuin`. * Environment variables can differ between the daemon and client. Atuin used to store the daemon socket in `$XDG_RUNTIME_DIR`; however, this caused issues for users whose daemons were launched in environments that did not have `XDG_RUNTIME_DIR` set, but whose shells did, as the client would fail to find the daemon socket. This would be less of an issue today, now that the socket path is stored in the daemon pidfile, however it is arguably incorrect that the pidfile is stored in `$XDG_DATA_HOME` since it is relevant only when a running instance of Atuin exists. But storing the pidfile in `$XDG_RUNTIME_DIR` would simply reintroduce the same issue. Additionally, when the socket was stored in `$XDG_RUNTIME_DIR`, the behavior of the daemon when `XDG_RUNTIME_DIR` was unset was not spec-compliant, as the daemon would fall back to storing it in `$XDG_DATA_HOME`. Being spec-compliant is not trivial, however: the XDG Base Directory Specification says that applications “should fall back to a replacement directory with similar capabilities”, but among the capabilities mandated of `$XDG_RUNTIME_DIR` are that “if the user fully logs out the directory MUST be removed”. It is not straightforward for Atuin to produce a fallback directory with this capability when `XDG_RUNTIME_DIR` is unset, and even if it could, it would not solve the daemon–client mismatch issue. * In our opinion, not all of the XDG Base Directories have the same level of discoverability as `~/.atuin`. This is particularly the case for logs, and it is why even today Atuin stores logs in `~/.atuin/logs`. The correct place for logs in the XDG Base Directory Specification is `$XDG_STATE_HOME`, falling back to `~/.local/state`, though note that `XDG_STATE_HOME` was added to the spec only in 2021. We do not think that our users would be as likely to check `~/.local/state/atuin` for logs as `~/.atuin`; many likely have not heard of `~/.local/state` at all. * Atuin's installer, [cargo-dist], is not set up to install to the XDG Base Directories; it is tailored towards following the same model as Cargo and rustup, with everything in a single directory. Even if Atuin's installer installed to `~/.local/bin`,[^3] this would complicate the setup of `$PATH`. We cannot rely on `~/.local/bin` being in the user's `$PATH`, so the installer, which currently adds `~/.atuin/bin` to `PATH`, would have to add `~/.local/bin` instead if it wasn't already there. But adding `~/.local/bin` is potentially a much larger and less isolated change: if that directory contains binaries but isn't in `$PATH`, adding it injects more than just `atuin` into the user's environment. [`dirs`]: https://docs.rs/dirs/7 [`cache_dir`]: https://docs.rs/dirs/7/dirs/fn.cache_dir.html [`config_local_dir`]: https://docs.rs/dirs/7/dirs/fn.config_local_dir.html [`data_local_dir`]: https://docs.rs/dirs/7/dirs/fn.data_local_dir.html [cargo-dist]: https://github.com/axodotdev/cargo-dist [^3]: It should be noted that `~/.local/bin` was added to the XDG Base Directory Specification only in 2021, and unlike every other directory in the specification, cannot be overridden with an environment variable. <h3 id="migrating-existing-installations">Migrating existing installations</h3> Migrating existing installations to the new directory layout is not required; they will continue to work as-is. However, if you wish to perform an explicit migration, you can do so by running these commands in Bash, Zsh, or fish (if you have `XDG_CONFIG_HOME` or `XDG_DATA_HOME` set, you will have to adjust the paths). Temporarily stop the daemon and disable autostart: ```sh atuin config set daemon.autostart false atuin daemon stop ``` Make sure `~/.atuin` exists: ```sh mkdir -p ~/.atuin ``` Check whether you have any AI-related files in `~/.atuin`: ```sh ls ~/.atuin | grep -Fxe permissions.ai.toml -e skills -e TERMINAL.md ``` If you do, please expand and follow this section: <details> <summary>Migrating existing AI config in <code>~/.atuin</code></summary> In older installations, `permissions.ai.toml`, `skills`, or `TERMINAL.md` in `~/.atuin` are scoped to your home directory, whereas in newer installations, they are applied *globally*. The existence of `~/.atuin/config.toml` is what determines whether the new or old behavior is used; if `~/.atuin/config.toml` exists, these items are interpreted as global. Before you move your `config.toml` to `~/.atuin`, review the existing AI files in `~/.atuin`, especially `permissions.ai.toml`. Anything you're ok with being made global can be left as-is (global is likely not too different from applying to your entire home directory anyway); anything else should be deleted or moved to subdirectories (meaning moving it into `~/subdir/.atuin/` for each subdirectory). After you've confirmed that the AI items in `~/.atuin` can be interpreted as global, check whether any AI items appear in both `~/.atuin` and `~/.config/atuin`: ```sh ls ~/.atuin ~/.config/atuin | grep -Fxe permissions.ai.toml -e skills \ -e TERMINAL.md | sort | uniq -d ``` If there are any such items, you must merge the items in `~/.config/atuin` into the configuration at `~/.atuin` if you still want them to take effect; otherwise, the items in `~/.config/atuin` will be shadowed by those in `~/.atuin`. </details> Next, move your config items: ```sh cd ~/.config/atuin && mv -n $(ls | grep -Fxe config.toml -e server.toml \ -e themes -e skills -e permissions.ai.toml -e TERMINAL.md) ~/.atuin/ ``` Then, move your data: ```sh [ -e ~/.atuin/data ] && echo 'error: destination exists' || \ mv -n ~/.local/share/atuin ~/.atuin/data ``` Finally, re-enable daemon autostart, if you had it enabled before: ```sh atuin config set daemon.autostart true ``` ### Implementation details This PR adds a new module, `atuin_common::dirs`, responsible for calculating the paths to all of Atuin's files and directories. See that module for details on how paths are calculated now. A priority of the module is to ensure that existing installations work without modifications, while encouraging new installations to move toward `$ATUIN_HOME`. `ATUIN_CONFIG_DIR` now officially affects `config.toml` and `server.toml` only. Previously, it affected the location of `config.toml`, `server.toml`, and `themes`, but not `skills`, `permissions.ai.toml`, or `TERMINAL.md`, which ignored the environment variable and were always looked up in `~/.config`. For backward compatibility, if `ATUIN_CONFIG_DIR` is set, `ATUIN_HOME` is unset, and `~/.atuin/themes` does not exist, themes will continue to be read from `$ATUIN_CONFIG_DIR/themes`. `ATUIN_THEME_DIR` is deprecated. For backward compatibility, if `ATUIN_HOME` is unset and `~/.atuin/themes` does not exist, themes will continue to be read from `$ATUIN_THEME_DIR`. `data_dir`, `db_path`, `key_path`, and `ai.db_path` are no longer mentioned in the example `config.toml` or the corresponding web documentation. The actual options have not been removed and are still respected. The rationale behind this change is that customizing these paths at that level of granularity is a highly advanced feature that the vast majority of users will not need and that can cause problems; listing them in the example config and documentation (at the very top, no less!) may encourage newer users to modify these settings unnecessarily. Note that several other options of this kind were already undocumented, including `record_store_path`, `kv.db_path`, and `scripts.db_path`; this change unifies all such options to the same treatment of being less visible to new users, but still respected internally. Fixed a bug where Atuin AI always read snapshots from `~/.local/share/atuin/ai/snapshots` (or the equivalent in `$XDG_DATA_HOME`), ignoring `data_dir` in `config.toml`. It now respects `data_dir`, but for backward compatibility, will still check the old path if `ai/snapshots` doesn't exist in `data_dir`. Fixed undefined behavior in tests due to the use of `std::env::set_var` and `std::env::remove_var`. A new module, `atuin_common::env`, has been added, which wraps `std::env::var` and `std::env::var_os` but can be mocked safely in tests. There is no performance penalty or behavior difference in normal builds; the mock functionality is compiled in only when `#[cfg(test)]` is true or when the `test-utils` feature is enabled (so it can be used in other crates' tests). Normal builds enable neither of these. In atuin-client, removed the global `DATA_DIR` variable that got initialized when constructing the first `Settings` instance. This was error-prone and made it hard to ensure that it would be initialized by the time consumers needed it, and obscured the dependency that some code had on `Settings`. Now, `Settings::data_dir` is the one true source to get the path to the data dir. Similarly to the `data_dir` change, accessing the meta store requires calling the `meta_store` method on a `Settings` instance instead of relying on a global variable that couldn't be guaranteed to be initialized when needed and could panic. As before, only one meta store is constructed per process, even if `meta_store` is called on different `Settings` instances. This matches the existing behavior but could be revisited if necessary. The behavior of `atuin_client::daemon::socket_path` is different when `ATUIN_HOME` is set. In this case, Atuin will not connect to an existing socket unless it matches the one in the pidfile (this also means the legacy socket paths aren't checked). `ATUIN_HOME` is being introduced after Atuin already switched to storing the socket path in the pidfile; thus, if there's no socket path in the pidfile but the socket exists, the socket is almost certainly that of a different program that we shouldn't connect to (potentially that of an Atuin daemon running with a different value of `ATUIN_HOME`). Similarly, when `ATUIN_HOME` is set, the `--force` flag for `atuin daemon start` will not remove legacy socket paths, since Atuin never creates or connects to those paths in this case. Atuin AI no longer offers "Always allow in this directory" or "Always allow in this workspace" when the current directory/workspace is `$HOME`, because files like `~/.atuin/permissions.ai.toml` are now interpreted *globally*. It would be possible to re-implement this functionality if necessary, by storing the `$HOME`-scoped files in a different path such as `~/.atuin/home/`, but this has been left out for now as it is not clear how necessary it is: configuration scoped to the user's entire home directory is, for most practical purposes, nearly equivalent to global configuration anyway. ### Important files changed/created * `crates/atuin-common/src/dirs.rs`: new module containing all the logic for determining which paths to use * `crates/atuin-daemon/src/client.rs`: new logic for determining the daemon socket path * `crates/atuin-ai/src/permissions/writer.rs` and `crates/atuin-ai/src/tui/view/mod.rs` have new logic when the current directory or workspace is `$HOME`. * `crates/atuin-client/src/settings.rs`: cleaned up `Settings` interface; no longer relies on global potentially-uninitialized variables * `docs/docs/configuration/config.md`: updated to mention `ATUIN_HOME` and provides optional migration instructions * `crates/atuin-client/src/theme.rs`: cleaned up theme manager logic * `crates/atuin-common/src/env.rs`: new wrappers around `std::env` to avoid UB in tests * `crates/atuin-common/src/os/unix/socket.rs`: new *non-blocking* utility for checking whether a socket is in use ### Limitations Note that even with this change, the install script will still create `~/.config/atuin/atuin-receipt.json` (or `%LOCALAPPDATA\atuin\atuin-receipt.json` on Windows). This path is hardcoded by [cargo-dist]. To change this, cargo-dist and axoupdater would need to add support for customizing the receipt path, or we would need to fork those crates, or we would need to switch to a different installer and updater. ### To-do before merge - [x] Smoke testing: - [x] Existing installs continue to work - [x] Fresh installs work and go in `~/.atuin` - [x] `ATUIN_HOME` works, including multiple profiles simultaneously - [x] Migration instructions work - [x] Partially migrated installs work (e.g., `data` only) - [x] New Atuin AI behavior for `$HOME` works