In SourceKit-LSP, we can get into the following situation:
1. We open A.swift
2. We issue a request for A.swift, the request takes a while to execute
3. The dependencies of A.swift are updated, which causes us to reopen the document in sourcekitd, so that the AST is rebuilt
4. This shouldn’t cause the request from (2) to be cancelled. We should continue executing it and only re-open the document after the request from (2) has finished
rdar://127475366
This allows us to run `sourcekit-lsp index --project /path/to/project` to index a project. Intended to debugging purposes, eg.
- Profile the time it takes to index a project
- See if the project can be indexed successfully
- Look at signposts generated during indexing in Instruments to see whether indexing or preparation is the bottleneck and how well we can parallelize tasks.
This allows a user of SourceKit-LSP to inspect the result of background indexing. This allows a user of SourceKit-LSP to inspect the result of background indexing. I think this gives useful insights into what SourceKit-LSP is indexing and why/how it fails, if it fails, also for users of SourceKit-LSP.
rdar://127474136
Fixes#1265
When the user opens documents from three targets A, B, and C in quick succession, then we don’t want to schedule preparation of wait until A *and* B are finished preparing before preparing C.
Instead, we want to
- Finish for preparation of A to finish if it has already started by the time the file in C is opened. This is done so we always make progress during preparation and don’t get into a scenario where preparation is always cancelled if a user switches between two targets more quickly than it takes to prepare those targets.
- Not prepare B because it is no longer relevant and we haven’t started any progress here. Essentially, we pretend that the hop to B never happened.
We were mixing the up-to-date status and in-progress status of an index task in `SemanticIndexManager`. This meant that a single `QueuedTask` in the task scheduler could be needed for eg. both preparation for editor functionality in a file of that target and to re-index a file in that target. This dual ownership made it unclear, which caller would be entitled to cancel the task. Furthermore, we needed to duplicate some logic from the preparation task dependencies in `SemanticIndexManager.prepare`.
To simplify things:
- Split the up-to-date status and the in-progress status into two different data structures
- Make the caller of `prepare` and `scheduleIndex` responsible for cancellation of the task it has scheduled. `TaskScheduler` might receive more scheduled tasks this way but the additional tasks should all be no-ops because the status is known to be up-to-date when they execute.
The logging from `TaskScheduler` was really verbose (and somewhat useful during its development) but it turns out that it’s just noise in real-world debugging. So, let’s remove it.
Essentially fix two issues in updating the index store:
1. If there was one task to index `HeaderA.h` through `main.c` and one to index `HeaderB.h` through `main.c`, we would not declare a dependency between them in the task scheduler, which meant that we could have two concurrent and racing index tasks for `main.c`. Declare a dependency between any two files that have the same main file
2. `UpdateIndexStoreTaskDescription` was computing the target to index a file in independently of `SemanticIndexManager`. While they currently always line up, we should pass the target in which to index a file to the `UpdateIndexStoreTaskDescription`. Only this way can we guarantee that we actually prepared the target that the file will be indexed in.
The existing ad-hoc logic was not quite correct because it didn’t eg. remove `-MT/depfile` because it assumed that `-MT` was followed by a space. It also didn’t take into account that `serialize-diagnostics` can be spelled with a single dash or two dashes.
Create a `CompilerCommandLineOption` type that forces decisions to be made about the dash spelling and argument styles, which should help avoid problems like this in the future.
We weren’t logging requests sent to a `TestSourceKitLSPClient` because we were assuming that `JSONRPCConnection` logs those requests in `SourceKitLSPServer`. But no logging happens in `LocalConnection`, which `TestSourceKitLSPClient` uses.
Since `LangaugeServerProtcol` can’t depend on `LSPLogging`, move the type to `LSPTestsSupport`.
Whenever we get request for a document, open it or edit it, trigger a preparation of its target, but don’t block any interaction based on it. This should ensure that the target is usually prepared when the user is interacting with it.
We need to track the preparation status of targets somewhat accurately in `SemanticIndexManager`, so we don’t unnecessarily re-prepare a target. When updating the index store, it is acceptable to schedule another `UpdateIndexStoreTaskDescription` because it will early exit based on an `mtime` check with the unit file. Null builds of a target take significantly longer and thus we want to avoid them.
Fixes#1252
rdar://127474003
When a header is modified, we don’t we want to re-index all main files that include it. Instead, we just want to index one main to effectively re-index the header itself.
I originally implemented re-indexing of all files that include the header but on second thought, headers are like Swift modules, where we also don’t re-index all dependencies either. And if you change a low-level header that’s included by the entire project, you probably don’t want the indexer to go off and re-index the entire project.
