This allows us to fix a toolchain when using a `SwiftPMBuildSystem`, which is critical to ensure that a target gets prepared using the same toolchain that is used to index it and that is used for sourcekitd.
This also means that you can use the index log to view which tasks are currently being executed.
Since we only have a single log stream we can write to, I decided to prefix every line in the index log with two colored emojis that an easy visual association of every log line to the task that generated them.
If the user has enabled background indexing in sourcekit-lsp but opens a project that doesn’t support background indexing (compilation database, build server), we should show a message after opening the workspace, informing the user that background indexing is only supported in SwiftPM projects at the moment.
Fixes#1255
rdar://127474711
When looking for a workspace that can handle a file, we were creating full-fledged workspaces along the way, which we would then discard if they couldn’t handle the file being opened. This had multiple problems:
1. When background indexing is enabled, it caused semantic indexing of the workspace, which wrote files to a `.index-build` directory and was a waste of work
2. When background indexing is enabled, it caused package resolution, which also created a `.index-build` folder to be created
3. It caused a syntactic test index of the workspace, which was a waste of work.
To fix this, do multiple things:
1. When creating a workspace, add a check right after build system creation. This allows us to early exit if the build system can’t handle the file and prevents us from generating the `Workspace`, fixing (1) and (3)
2. Don’t call `reloadPackage` when creating a `SwiftPMWorkspace`. Instead, explicitly call `generateBuildGraph` once we committed to creating the workspace.
This allows a user of SourceKit-LSP to inspect the result of background indexing. This allows a user of SourceKit-LSP to inspect the result of background indexing. I think this gives useful insights into what SourceKit-LSP is indexing and why/how it fails, if it fails, also for users of SourceKit-LSP.
rdar://127474136
Fixes#1265
Instead of asking for build settings of a file, the build system manager asks for the targets of a file and then asks for the build settings of that file in a specific target. This has two advantages:
- We know about targets and can prepare the targets for background indexing
- Once we support build systems in which a single file can be part of multiple targets, we can have a centralized place that picks preferred targets for a file, eg. based on user configuration
This workspace-wide syntactic test index is used for two purposes:
- It is used for XCTests instead of the semantic index for files that have on-disk or in-memory modifications to files
- It is uses for swift-testing tests, which are only discovered syntactically.
rdar://119191037
When we couldn’t start a build server or find a SwiftPM package, we currently always create a `CompilationDatabaseBuildSystem`, even if no `compile_commands.json` or `compile_flags.txt` exits. Every request for build settings would then log an error that the compilation database can’t be opened, which was very spammy. Instead, if the compilation database can’t be loaded, just set the build system to `nil` and log a single error message.
OSLog is the suggesting logging solution on Apple platforms and we should be using it there, taking advantage of the different log levels and privacy masking.
Switch sourcekit-lsp to use OSLog on Apple platforms and implement a logger that is API-compatible with OSLog for all uses in sourcekit-lsp and which can be used on non-Darwin platforms.
The goal of this commit is to introduce the new logging API. There are still improvements about what we log and we can display more privacy-insensitive information after masking. Those changes will be in follow-up commits.
Add `.swift-format` to the repo and format the repo with `swift-format`.
This commit does not add any automation to enforce formatting of sourcekit-lsp in CI. The goal of this commit is to get the majority of source changes out of the way so that the diff of actually enforcing formatting will have fewer changes or conflicts.
The core idea here is that the toolchain language servers always call into `BuildSystemManager` and `BuildSystemManager` will always reply with build settings. If it hasn’t computed them yet, it will reply with fallback settings.
With that assumption, we can remove the `documentToPendingQueue` from `SourceKitServer` since there are no longer any documents that are pending – everything has a build settings immediately.
Similarly, `BuildSystemManager.mainFileStatuses` also isn’t needed anymore.
And lastly, since we know that `BuildSystemManager.buildSettings` will always return a value `registerForChangeNotifications` is changed not call `fileBuildSettingsChanged` immediately. Instead, it will only cause `fileBuildSettingsChanged` to be called when the file’s build settings change after the `registerForChangeNotifications` call.
This defines away an entire class of data races if delegate callbacks are delivered out-of-order. If we aren’t providing the new build settings in the delegate callback, then it doesn’t matter if two `fileBuildSettingsChanged` calls change order since they don’t carry any state.
Instead of storing build settings inside the language servers based on update notifications from the build system, always call into the `BuildSystemManager` to get the build settings.
Overall, I think this is a much clearer separation of concerns and will allow us to remove `SourceKitServer.documentToPendingQueue` in a follow-up commit as `SwiftLanguageServer` can always directly call into `BuildSystemManager` to get build settings and we don’t need to wait for the initial notification to receive the first build settings.
This requies `BuildServerBuildSystem` to keep track of the build settings it has received from the BSP server.
`ClangLanguageServer` still caches build settings locally. `ClangLanguageServer` will change to the same pull-based model in a follow-up commit.
The asyncification changes caused some non-deterministic test failures. I believe that some of these are due to race conditions that are the result of the partial transition to actors.
Instead of merging the asyncification piece by piece, I will collect the changes asyncification changes in a branch and then qualify that branch througougly (running CI multiple times) before merging it into `main`.
Instead of storing build settings inside the language servers based on update notifications from the build system, always call into the `BuildSystemManager` to get the build settings.
Overall, I think this is a much clearer separation of concerns and will allow us to remove `SourceKitServer.documentToPendingQueue` in a follow-up commit as `SwiftLanguageServer` can always directly call into `BuildSystemManager` to get build settings and we don’t need to wait for the initial notification to receive the first build settings.
This requies `BuildServerBuildSystem` to keep track of the build settings it has received from the BSP server.
`ClangLanguageServer` still caches build settings locally. `ClangLanguageServer` will change to the same pull-based model in a follow-up commit.
Instead of storing build settings inside the language servers based on update notifications from the build system, always call into the `BuildSystemManager` to get the build settings.
Overall, I think this is a much clearer separation of concerns and will allow us to remove `SourceKitServer.documentToPendingQueue` in a follow-up commit as `SwiftLanguageServer` can always directly call into `BuildSystemManager` to get build settings and we don’t need to wait for the initial notification to receive the first build settings.
This requies `BuildServerBuildSystem` to keep track of the build settings it has received from the BSP server.
`ClangLanguageServer` still caches build settings locally. `ClangLanguageServer` will change to the same pull-based model in a follow-up commit.
Explicitly import interfaces from TSCBasic which now allows us to
identify all the swift-tools-support-core interfaces which are in
use in SourceKit-LSP.
This allows sourcekit-lsp to make use of the path remappings recently added to
the index store and IndexStoreDB to remap remote paths into local paths
when loading index data locally.
These remappings can be provided via the `-index-prefix-map` command line flag to sourcekit-lsp or via the `BuildSystem` integration point.
This sets the stage for multi-workspace support. Everything should be handled internally, we are just missing the API to actually open multiple workspaces at this point.
This sets the stage for multi-workspace support. Everything should be handled internally, we are just missing the API to actually open multiple workspaces at this point.
Similar to how we reload the package if Package.swift is changed, we also watch `compilation_database.json` and `compile_flags.txt` and notify the build system delegate that build settings changed if these files are modified.
rdar://92388223 [#486]
Implement rudementary support for `DidChangeWatchedFileNotification` for SwiftPM projects: When a file is added, reload the Swift package to compute build settings for it.
This enables proper semantic functionality added to the project after the LSP server was started.
Resolves SR-15633
Remove `settings(for:)` from the BuildSystem API in favor of the change callback
- With the `settings(for:)` method removed, we instead block on a
callback from the `BuildSystemManager` which itself handles
the primary build system callback in conjunction with the fallback
system
- We queue up notifications / requests that reference documents
until we receive the callback.
- Remove language from FileBuildSettings as it is no longer needed,
Swift-specific working dir fixes are now in SwiftCompileCommand
- registerForChangeNotifications should inform its delegate
asynchronously instead of immediately
In preparation for injecting doing more during registration, pass
through the same parameters (add language) so that we can call settings
as necessary.
Find the index store path by searching through the command-line
arguments, and if found, also provide a default database path next to
the index store. Also add command-line arguments so that either of these
can be overridden. We could also easily add these as initialization
options if an LSP client wanted to provide them in the future.
URL can in fact store URIs, it just doesn't have a very nice API to
interact with them. As long as we only operate on absoluteString, we
should be fine though. So instead of implementing the logic for
detecting file URLs ourselves, we can just use a URL as storage for
DocumentURI.
According to the LSP specification, arbitrary URIs can be used as
document identifiers. Instead of internally assuming that all URIs are
URLs, use a DocumentURI enum to represent URIs. These can either be file
URLs or other URIs whose value as treated as an opaque string.
